How to Run Multiple Commands Simultaneously in Linux
You may know the how to use
& to run commands in the background, but it also has this other very interesting use case.
We can run two or more commands asynchronously using the single ampersand
command1 & command2 & command3 ...
Suppose we had three scripts we wanted to run simultaneously.
./s1.py & ./s2.py & ./s3.py
All three Python scripts will run simultaneously in separate sub-shells, allowing you to take advantage of multiple cores. The output of all three, however, will all be attached to this parent shell. This means each line will be displayed on a first-come, first-serve basis.
If you’re having trouble running Python scripts through bash like this, you may have forgotten to add
#!/usr/bin/env pythonto the top of your script. Another alternative is just to run the scripts through the
If you need to examine the output of each separately, it may be helpful to redirect each script output to a file.
./s1.py > log1.txt & ./s2.py > log2.txt & ./s3.py > log3.txt
Because this is asynchronous, none of these commands should depend on one another.
For example, this string of commands would cause issues.
./s1.py > out1.txt & grep "some_text" out1.txt
In the second command, we are looking for the text
some_text in the output of the first command, which doesn’t exist yet.
This is why we may need synchronous commands.
We can run two or more commands synchronously using double ampersands
command1 && command2 && command3 ...
One thing to note with
&& is that each subsequent command is dependent on the success of the previous command.
command2 will never run. If
command3 will never run, and so on .
Continuing with our previous example:
./s1.py > out1.txt && grep "some_text" out1.txt
This is a valid command using
grep "some_text" out1.txt will run as long as
./s1.py doesn’t contain any errors.