Posted by CANbike on Mon, 19 May 2014

sed: Delete Carriage Returns and Linefeeds (CRLF)

A carriage return linefeed (CRLF) is a special sequence of characters, used by DOS/Windows, to signify the end of a line of text. However, in *nix, only a linefeed (LF) is required to signify a line ending.


Delete/Replace a Carriage Return (CR) with sed

There are a few ways to remove or replace a carriage return with sed.

  • sed 's/.$/<replace with text>/'
  • sed 's/\x0D$/<replace with text>/'
  • sed 's/\r/<replace with text>/'

The first method assumes all lines end with CRLF, while the second and third method depend on the version of sed.


Delete/Replace a Linefeed (LF) with sed

By default, sed strips the newline as the line is placed into the pattern space. However, the workaround is to read the whole file into a loop first.

sed ':a;N;$!ba;s/\n/<text>/g'

where

:a       - create a label 'a'
N        - append the next line to the pattern space
$!       - if not the last line
ba       - branch (go to) label 'a'
s        - substitute
/\n/     - regex for new line
/<text>/ - with text "<text>"
g        - global match (as many times as it can)


Example: Delete Newlines (CRLF) with sed

Delete the carriage return first, and then delete the linefeed.

  • example.txt
    Example.png
  • sed -i 's/\r//g' example.txt
    sed example delete newline carriage return.png
  • sed -i ':a;N;$!ba;s/\n/<text>/g' example.txt
    sed example delete newline linefeed.png


tr: A Simpler Way to Replace or Delete a Newline (CRLF)

The tr utility is a preferred and simpler method for replacing end of line characters as the pattern buffer is not limited like in sed.

tr '\r\n' '<replace with text>' < input_file > output_file

or to delete the newline (CRLF),

tr -d '\r\n' < input_file > output_file

where

\r - carriage return
\n - linefeed


Example: Delete Newlines (CRLF) with tr

  • example.txt
    Example.png
  • tr -d '\r\n' < example.txt > example2.txt
    tr example delete newline CRLF.png


Command Line Examples

Note: The command cat will concatenate files and print on the standard output. The option A will use ^ and M notation to show non-printing characters, display $ at end of each line, and display TAB characters as ^I.

Terminal examples to delete CRLF newline.png


Related Item(s):