/bin/sh Scripting SFTP transfers

Recently I got lumped with scripting file transfers to a third party via SFTP.

The only information I was provided was that I needed to create a user and that an undetermined number of files would be dumped once a week into this users directory which in turn needed to be SFTP’ed elsewhere. The only two things I had to go on with the files was that the time it was supposed to occur as well as the fact that they ended with the same prefix which was yyyymmdd.txt and related to the date the files were generated.

The first thing I did was to organise a SSH key pair between my user and the remote SFTP user to remove the need for passwords in the script. A quick Google brings up this how-to if you are curious on how to do this part.

I used the batch option of SFTP which is designated as -b and is a file of all the FTP commands you want to send to SFTP. I built this file via a simple for do loop based on the date it ran.

F_EXTENSION=`date +”%Y%m%d”.txt`

for SENDFILE in `ls *${F_EXTENSION}`
echo “put ${SENDFILE}” >> sftp_batch.$$

Once the batch file has been generated it is just a matter of calling SFTP to transfer the files.

sftp -b sftp_batch.$$ user@ftp.server.com: 2> sftp_error.$$ 1> sftp_log.$$

NB: The above is supposed to be one line if you are seeing it spread over multiple lines.

For my script, I needed to know if it was successful or not and act accordingly. To do this, I pushed the SFTP output of standard error to one file and standard out to another. If standard error was >0 then an error had occurred which was checked later on in the script and generated the appropriate email to those who needed to know.