LibGuides: Chemistry 470: Thesis Data Management

Data Storage and Backup

Keep multiple copies: two copies of your data, in two different locations, one of them on the cloud (Reed’s Google Drive) unless not allowed due to data size or privacy. Email data@reed.edu if you think you might have special circumstances

Backup your data regularly. Backup recommendations from Reed.

Regularly audit your backups: are they actually happening? Are they happening the way you intended??? Check for expected number of files, matching file size between copies, can the files be opened.

Keep the 10 Steps to Safer Computing in mind

Folder and File Management

Consistency is key!
Pick a system and stick with it. It needs to make sense to you so that you will use it.
Follow a consistent folder system for storing data - ideally one that makes sense without deep knowledge of the project.
Document your system in your lab notebook
Do not rely on the folder structure to provide critical context for files. Files copied elsewhere will lose this information.
Avoid spaces and characters other than - _ (dash and underscore) in your filenames - - not all software applications can handle them and they might get changed by the system
For dates, use the format YYYYMMDD, YYYY-MM-DD, or YYYY_MM_DD. With this date format, files sorted alphabetically will also sort chronologically.
Keep track of different file versions by using a suffix to represent the version number.
- Example: ProjectName_Instrument_Condition_YYYYMMD_v01.txt
Decide in advance what kind of changes will render a version change

Use this	Instead of this
Project1/SiteB/SiteB_2021-09-15_rawdata.txt	Project1/SiteB/2021/09/15/rawdata.txt
sequence_readings_2021-09-15.csv	sequence_readings_3rd_sept.csv
SiteB_2021-09-15_processed_v02.r	SiteB_2021-09-15_processed_redo.r

Data Documentation

Document the decisions you make about your data during the research process. It is easy to forget and you may need to refer to it later during the thesis writing process.

Describe the content of your data files in a data dictionary. List variables with definitions, units of measure, scope notes, coded values. Document how missing values are represented. List the file formats you are using.

After your thesis is handed in

If possible, migrate your data out of software specific formats to an open format (such as a csv). This is especially true for files that require an expensive and specailized software program to open - will you still be able to access your data after you graduate?

Consider uploading the data to the Reed College Dataverse

Data Analysis

Data @ Reed offers R Resources help. Still need help? Send an email to them team, data@reed.edu