As part of a conference I am currently organizing, we use the site Formstack to gather information from speakers like their bios, paper titles and such. Formstack includes the ability for user to upload files to a form. In our case, we included the option for speakers upload photographs of themselves for inclusion in the program.

Formstack does not have an automated way to export these files via their user interface as far as I am able to tell. When form submissions are exported from Formstack as an Excel or CSV file, the uploaded files are provided via a link to the Formstack site from which you can download the file (provided you are logged in). When you have hundreds of files to download, this becomes a pain in the butt. The other issue is that the images are named with a unique ID appended to the original file name, making it hard to for us to match the image file to the speaker. And of course, speakers themselves don’t use consistent file names either.

I wanted to figure out a way to automatically download and rename the photos, and after some sleuthing and probably more time taken than just doing it manually, I eventually figured out a simple method. This is a pretty niche use case but it might be helpful for others struggling with a similar set of requirements. I could see this being applicable to lots of needs outside of Formstack too. The most useful resource I found when coming up with this process/script was the Stack Overflow post “How to get past the login page with Wget?”.

A. Relevant systems stuff you’ll need:

• The Mac or Linux terminal or some equivalent
• wget installed
• Google Chrome

B. Format the links CSV file

The first thing to do is create a CSV file with the list of links in the first column and the intended output filename in the second column. This can easily be done by exporting the form submissions from Formstack in spreadsheet format, deleting all but the columns you need, and creating a concatenated column based on the speaker’s name, for example. Re-save the file in CSV format, but not with Excel’s UTF-8 encoding format option since this will add a BOF file marker you would have to remove.

https://www.formstack.com/admin/download/file/5959596969,GeorgeBrown.jpg
https://www.formstack.com/admin/download/file/5959596970,MariaSmith.jpg

In my case I wanted all files renamed with the .jpg extension. If there are multiple types of file formats expected then it might be trickier, requiring a mime-type check or the like. Formstack allows you to limit uploads by file by extension so that might be something to consider when designing the form too.

One thing to check with your CSV is that if you are using Excel, it will add a carriage return character (\r) or 0D in hex to the ending of lines. You will need to get rid of this so that the text file can be processed using your bash terminal.

There are various ways to do this (See “Remove carriage return in unix”) but what worked for me is just running gsed 's/\r$//' infile.csv > outfile.csv (I use the gsed tool since Mac’s version of sed is not as full-featured).

B. Get cookies file

Next, install the Chrome extension Get Cookies.txt. This handy tool will allow you to export the cookies.txt file for a browser session. This file will essentially give permission to your terminal to access the website as a logged-in user.

Log in to Formstack and export the cookies file. Click on the Get Cookies.txt extension and then ‘Export’ to download the file. It should have the filename formstack.com_cookies.txt Note that you will need to download a new cookies file for each new logged-in session in Formstack; if you have logged out of Formstack then you will need to log back in and re-export the file.

C. Run the script

1. Put the csv file and the formstack.com_cookies.txt file in the same folder.

2. Create a text file, copy and paste the script below, and edit it so that the path to the cookies file is correct in line 3 and the filename of your CSV matches in line 4.

while IFS=, read -r field1 field2 || [ -n "$field1" ] ; do
	echo "$field1 and $field2"
	wget --load-cookies /Path/To/formstack.com_cookies.txt "$field1" -O "$field2"
done < links.csv

3. Save the script with a .sh extension. Name the script whatever you like.

4. In the terminal, cd into the folder you created and then run the script by running bash script.sh, with script.sh being the name of your bash script file.

5. The script should then start downloading your files from Formstack, renamed all nice!

Back to posts