search

edbennett / sp2n-topology-202205

MIT License
0 stars 0 forks source link

Package data #5

Closed edbennett closed 2 years ago

edbennett commented 2 years ago

We need to have the data packaged to upload separately to Zenodo. This needs:

edbennett commented 2 years ago

@dvadacchino Do you have the raw output files from the flow? An important principle of sharing data (as I see it at least) is keeping raw data raw—each processing step is a step that others can’t reproduce, and a possibility for an unauditable error to be made.

dvadacchino commented 2 years ago

Sure, I will also add to the repository the script I am using to format the raw output files

edbennett commented 2 years ago

Thanks, I see it, and will incorporate it into the Makefile since it exists now.

Could you put the files into the shared /scratch on Sunbird please?

dvadacchino commented 2 years ago

Done, look at /scratch/scw1019/DV/spN/raw

edbennett commented 2 years ago

Thanks—it looks like at least two files are missing: out_4_20_7.72 and out_2_24_2.62

edbennett commented 2 years ago

I've found /scratch/scw1019/DV/spN/sp4/nc4_b7.72_20/WF/out_4_20_7.72, but I don't see any sign of a Nc=2 L=24 beta=2.62 run on Sunbird?

dvadacchino commented 2 years ago

Sorry, I put the one you found in the raw directory, but not the other one, since it is never used in the analysis :)

dvadacchino commented 2 years ago

Moreover, I had to get rid of many things because of quota related issues, and it might be that 2.62 was included in the purge

edbennett commented 2 years ago

Ah, I see—I'll remove it from DATA_FILES then. (make was failing because I listed it as a dependency and then it couldn't find it.)

edbennett commented 2 years ago

@dvadacchino out_8_16_26.5 is malformed—configuration 2945 flows to t=5.75, then loops back to t=1.90. Any idea why?

edbennett commented 2 years ago

Another issue—out_4_20_7.72 has 301 steps for the configurations up to 2002, and then 251 steps for the subsequent configurations. This flow is also the only one to have used the Euler integrator rather than the RK3.

edbennett commented 2 years ago

I've resubmitted jobs for 4_20_7.72 on Sunbird just in case we don't have a version of this with the RK3 integrator.

dvadacchino commented 2 years ago

Actually you are right, the raw files seem to be gone. However, the file WF_2_20_7.72 was computed with the RK. What might have happened is that I involuntarly erased the new raw file instead of the old one...

edbennett commented 2 years ago

I did spot that out_4_20_7.72 and out_4_20_7.72_old (modified in February) were identical…

dvadacchino commented 2 years ago

Yes, I just copied the _old version, thinking it was the right file. But the file that produces WF_4_20_7.72, well, I can't find it

dvadacchino commented 2 years ago

Actually, its the same numbers. So I think this is a solid mistake on my part!

edbennett commented 2 years ago

Good thing we're doing this process so we could spot and fix that error. Do you know what happened with out_8_16_26.5? (Should that one malformed configuration be re-run?)

dvadacchino commented 2 years ago

Yes, thank you for spotting the mistake.

I think that the configuration in out_8_16_26.5 was malformed because of some sort of crash or because of a too short wall time. I will check in the raw file: it could be that the analysis was restarted from the same configuration, and 2945 can simply be discarded. The statistical effect of discarding it would be negligible, anyways.

edbennett commented 2 years ago

The replacement run is complete. I've now found out_8_16_27.0 has a similar issue—t=0.20 is repeated for configuration 2999.

edbennett commented 2 years ago

All documentation and code is ready, just need the final data now and this task can be closed.

edbennett commented 2 years ago

Closed by b54af3f

dvadacchino commented 2 years ago

added the first, the second is not used in the paper

On Sun, May 22, 2022 at 12:31 AM Ed Bennett @.***> wrote:

Thanks—it looks like at least two files are missing: out_4_20_7.72 and out_2_24_2.62

— Reply to this email directly, view it on GitHub https://github.com/edbennett/sp2n-topology-202205/issues/5#issuecomment-1133783280, or unsubscribe https://github.com/notifications/unsubscribe-auth/AS7CHWN7BZKMGEOCAW5EAATVLFW45ANCNFSM5WPMC3EQ . You are receiving this because you were mentioned.Message ID: @.***>