AMP-SCZ / lochness

Download your data to a data lake.
Other
3 stars 1 forks source link

Possible template errors #62

Closed tashrifbillah closed 2 years ago

tashrifbillah commented 2 years ago
  1. Shouldn't the first one be raw?

https://github.com/AMP-SCZ/lochness/blob/759298eace405621a4091642f864431924b999f9/lochness/tree/__init__.py#L26-L29

  1. Please double check these two templates:

https://github.com/AMP-SCZ/lochness/blob/759298eace405621a4091642f864431924b999f9/lochness/tree/__init__.py#L46-L49

https://github.com/AMP-SCZ/lochness/blob/759298eace405621a4091642f864431924b999f9/lochness/tree/__init__.py#L74-L77

I was expecting to see hearing and mindlamp words in them.

tashrifbillah commented 2 years ago

Also **kwargs should be removed from all of below:

https://github.com/AMP-SCZ/lochness/blob/759298eace405621a4091642f864431924b999f9/lochness/tree/__init__.py#L97-L118

kcho commented 2 years ago

1. / 2. We are not using behav_qc or hearing datatype, and they are left as they are. Mindlamp goes under phone datatype. We can also change this folder name, which we need to discuss with Justin.

kwargs

If you don't see any use of kwargs, I'm up for removing this from our functions.

tashrifbillah commented 2 years ago

Can you answer my question #1?

kcho commented 2 years ago

Can you answer my question #1?

We are not using behav_qc datatype for our U24 project and I think it's safe to ignore this in our AMP-SCZ/lochness. What do you think?

tashrifbillah commented 2 years ago

Okay thank you.

tashrifbillah commented 2 years ago

For #2, if hearing and mindlamp go to their respective folders, then the following:

https://github.com/AMP-SCZ/lochness/blob/4a707f045a32c3e6d16efffddd0810f6d6a368eb/lochness/tree/__init__.py#L94-L109

just reduces to:

if 'raw' in Templates[data_type]:
    # restructure root
    base = Path(base).parent.parent / study / 'raw'
    raw_folder = base / phoenix_id / Templates[data_type]

if 'processed' in Templates[data_type]:
    # restructure root
    base = Path(base).parent.parent / study / 'processed'
    processed_folder = base / phoenix_id / Templates[data_type]

That said, we should push for hearing and mindlamp to go to their respective folders for the simplicity of coding.

kcho commented 2 years ago

That said, we should push for hearing and mindlamp to go to their respective folders for the simplicity of coding.

  1. We are not using hearing in AMP-SCZ project and this does not affect our pipeline.
  2. For mindlamp, are you suggesting the mindlamp data to store data under mindlamp folder rather than the phone folder?
tashrifbillah commented 2 years ago
  1. Yes.
kcho commented 2 years ago

I know Habib already started working on his pipeline according to phone directory name, we should confirm if this change will affect any other pipelines with the group. I have brought it up in the DPACC slack channel.