Open codercahol opened 5 months ago
Further info:
the wells that cause the issue are consistent, with the following indices: [4608, 8065, 16130, 18819, 22276, 34181]
in the erroring code, the measurement times are all null, num_frames
is 2x that of the working code, and the threshold values do not match
Hey, I've just had a quick read. If I've understood correctly this is a code issue, and not an issue with the underlying data?
It's not immediately apparent to me why there should be a difference in execution depending on which functions are imported and run in the scope of the notebook. If it works correctly when the sub-functions are manually imported then that would suggest to me that there may be a name collision in the scope of the notebook? But I would expect that to cause the code to catastrophically fail rather than this subtle difference in a few rows.
In general it could be dangerous to import these main scripts as modules, because everything not wrapped in a if __name__ == '__main__':
clause will be executed at the time of import. Not sure if that could be the cause..?
Have you checked if all the other rows in exptl_df are identical in both methods of running it? Also, just running the main.py script directly from the terminal is how I used the code when I was working on it, is it possible to try that method on the cluster?
Sorry I don't have enough time this week to try running anything myself!
I get inconsistent results when I execute the code in
database_creation
:The following is all being run on the Carnegie cloud in a jupyter notebook
I execute the following to initialize the jupyter notebook:
the functions from the module are exported as follows:
When I run the
database_creation/main.py:main()
by executingcp.database_creation()
orcp.db_main.main()
I get an error indatabase_creation/main.py:merge_plate_and_well_info_dfs()
saying that thei
andj
columns of the dataframe areNaN
s.When I run all the sub-functions of
main()
in the notebook it runs to completion with no error (i.e. I define and runlocal_main()
as below):