Running:
---------
To enable the virtual environment, run
$ source env/bin/activate

To download the latest CDFs and create 1min averages, run
$ step1.py [year]-[month as number]

To create the final CDFs after offset files are generated by researcher, run
$ step2.py [year]-[month as number]

Additional options can be described by adding the --help flag.


Example:
--------
To run step1.py for June of 2020 for STEREO A,
	step1.py 2020-06
To run step2.py for February of 2014 for STEREO B,
	step2.py 2014-02 -sc=B


Description:
------------
step1.py downloads the L0 CDFs from Berkeley and creates 1 minute averages
directly from these files. It takes a year and month as its parameters. After
this is run the researcher, currently Hanying, should be notified to generate
the offset files, located in the folder named offset_files.

step2.py runs the processing script, correct.py, on each day in the month, and
does the job of finding which files correspond to which days since sometimes
there may be additional burst data files for a given day. It takes a year and
month as its parameters. Once this job is completed, the finalized CDFs and
flat files should be placed in the archive.


Structure:
----------
The structure of the scripts are as follows
	step1.py [year]-[month]
		download.py
		avg.py [l1 cdf filepath]

	step2.py [year]-[month]
		update_offset_tables.py [input offset file]
		correct.py [l1 normal cdf filepath] [optional l1 burst cdf filepath]
			timing_corr.py
			correct_data.py
			split_datarate.py
			add_offsets.py
			coord_transf.py
			avg.py 


Moving/Rebuilding:
------------------
If the scripts need to be moved to a new location, the following steps must
be performed.
1. Re-download the kernels by running:
   $ download_spice.py.py
   (Kernels are downloaded from SolarSoft's spice kernels tree.)

2. Re-install libraries and set up virtual environment by runnning:
   $ python3 -m venv env
   $ source env/bin/activate
   $ pip3 install -r requirements.txt

Alternatively, one can run setup/setup.sh to run all these steps at once.


Notes about the other files / folders
--------------------------------------
offsets/ - Offset files used in calculations
input_offsets/ - Input offset files

download.py [start year] [month] [day] [end year] [month] [day]
 	Downloads the L1 unvalidated files from Berkeley
avg.py [start year] [month] [day] [end year] [month] [day]
 	Creates 1 second averages used to generate offset file
update_offset_tables.py [offset_file]
 	Restructures a copy of the offset file and adds offsets to yearly file
process.py [cdf file]
	Fully corrects the L1 unvalidated CDF file and generates all
	the expected output files (burst, normal, 1s avg, 1min avg),
	(RTN vs SC coordinates)

constants.py - Common code / constants used by all the processing scripts
timing_corr.py - Code for adjusting timing bugs
add_offsets.py.py - Code for adding offset values from table to data
split_datarate.py - Code for creating 125ms avgs from 31ms data in mixed data
correct_data.py - Code for correcting the data and wrappers for stereoalg

Input and output data locations
-------------------------------
input_offsets - Location of input offset files before reformatting
offsets - Location of final offset files passed to scripts
l1_0_cdf - Location of input L1_0 CDF files from Berkeley
l1_0_avg - Location of 1-second averages from l1_0 cdfs
l1_final_cdf - Location of final L1 CDF files
l1_final_flat - Location of final L1 flat files
