Tuesday, November 13, 2007

Cannot execute [Argument list too long]

Tried to remove some files like log files with /bin/rm -rf all at once, and you get this
Error message

# rm -rf /var/log/mail/*.old.log
bash: /bin/rm: /bin/rm: cannot execute [Argument list too long]


So, rm utility complains that the system-wide ARG_MAX value that is used to setup an input buffer size to process the entire list will overflow. Good security measure, but, doesn't help you out with the task at hand.

To get around, use a combination of find, UNIX pipe, and xargs utilities. The rewrite of the original command would look something like this:

# find . -name '*.old.log' -print0 | xargs -0 rm -f

find naturally finds the target file names, and feeds them one by one into the unnamed UNIX/Linux pipe. The -print0 argument instructs to print the full file name on the standard output which is going to the pipe, followed by a null character (instead of the newline character that
-print̢۪uses
). When pipe becomes full, find blocks waiting for more space in the pipe to become available.

On the other end of the pipe, xargs command reads the next filename from the pipe, and executes the command specified with the filename parameter as an argument. The -0 argument instructs xargs to input items as terminated by a null character instead of by whitespace, and the quotes and backslash are not special (every character is taken literally).

getconf utility shows the system settings that most UNIX utilities rely on when allocating various resources including the memory buffers (such as ARG_MAX in this case).

# getconf ARG_MAX
131072


See http://home.comcast.net/~3rdshift/articles/linux_tips.html
for more

No comments: