Month: June 2019

#!/bin/bash
max_jobs=${JOBS:=$(nproc)}
pipe_file=$(mktemp --dry-run)
mkfifo ${pipe_file}
exec {pipe_fd}<>${pipe_file}
rm ${pipe_file}
printf "%-${max_jobs}s" >&${pipe_fd}
 
for job in ...
do
  read -n 1 -u ${pipe_fd}
  (
    echo "start ${job}"
    # do stuff for ${job} ...
    echo "finish ${file}"
 
    printf '%-1s' >&${pipe_fd}
  )&
done
wait

This piece of script uses a FIFO to manage the parallel jobs. Initially, the FIFO is filled with n characters, each representing an available slot for a job. Before a job starts, it reads a character from the FIFO. Once a job finishes, a character is written back to the FIFO. Thus the FIFO is empty if and only if there are n jobs running. New jobs will not be spawned because the FIFO read will block until at least 1 running job finishes.

{varname} style automatic file descriptor allocation requires bash 4.1+.

Reference:
https://superuser.com/questions/184307/bash-create-anonymous-fifo