COMBINE MULTIPLE CSV FILES FROM THE COMMAND LINE
For this guide we will take thousands of csv files and combine them into One single file, which will be uploaded to a database. In scraping sites, often my output files are separated and after i clean them up I need to combine them. This is how I do it.
On my Windows 10 computer
I will copy my CSV files over to a secondary drive, leaving the originals intact.
Then I will open a command prompt as an administrator and navigate to that drive. If the files are on my H: drive will navigate by typing
H:
Then press Enter
Now I will navigate to the “dataset” folder
cd dataset
I will then list the directory contents to ensure this holds my csv files
dir
Once I see all of my csv files I will enter the following
copy *.csv combined-date.csv
Then press Enter
This will create a file called combined-04132021.csv (which is today’s date)
And I can then upload that file to my database easily
