gammapy / gamma-cat

An open data collection and source catalog for gamma-ray astronomy
https://gamma-cat.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
15 stars 17 forks source link

Add info.yaml files in input/data #155

Closed pdeiml closed 7 years ago

pdeiml commented 7 years ago

I added two keywords to dataset_source_info.schema.yaml: data_entry_status: {missing, incomplete, complete}, required keyword review_required: {yes, no}, optional keyword

For existing files I decided to add the keywords with the following logic:

I would like to proceed as follows: When adding new data to any yaml-file and you think it is perfect (hence, without TODO remarks), then write 'review_required: yes' and @cdeil all files that you have reviewed you can mark with 'review_required: no' to not get confused about what you already reviewed.

Hence, grepping 'incomplete' in /input/data will show all files with something to do in it (with TODO remarks), grepping 'missing' will show files without any kind of data, grepping 'review_required' will show files which need to be reviewed.

cdeil commented 7 years ago

Just to note what we discussed on Skype just now. Suggest to add info.yaml in each paper folder with

data_entry:
  status: bla
  review_required: bla

Other useful info we could add to info.yaml:

As well as a list of available files in the folder under a datasets key.

Schema can come a bit later, when I also add the processing of info.yaml to the scripts and website generation.

pdeiml commented 7 years ago

Review requested. The second commit fixes two inconsistencies in datafiles which I found during the work.

cdeil commented 7 years ago

@pdeiml - Thanks!

The first commit ( 6d575e3 ) contains a lot of good additions. I'm reviewing the diff and doing a few follow-up commits fixing some mistakes (mostly accidental edits of existing YAML files).

As far as I can see, the changes in the second commit ( 3e41ac5 ) are incorrect, but I'm not 100% sure. If you want, please double-check against the latest version in gamma-cat master against the paper when you're back from vacation, and re-open a new PR proposing the change / fix if you think there's something wrong.

I'm closing this PR and will open a new one later today (to get rid of 3e41ac5 ).

cdeil commented 7 years ago

Continued in #159