hzovaro / spaxelsleuth

A package for analysing data from large integral field unit surveys such as the SAMI and Hector Galaxy Surveys.
MIT License
1 stars 1 forks source link

Bug in make_s7_df() when nthreads > 1 #12

Closed hzovaro closed 10 months ago

hzovaro commented 1 year ago

Running the following:

if __name__ == "__main__":
    from spaxelsleuth import load_user_config
    try:
        load_user_config("/Users/u5708159/Desktop/spaxelsleuth_test/.myconfig.json")
    except FileNotFoundError:
        load_user_config("/home/u5708159/.spaxelsleuthconfig.json")
    from spaxelsleuth.loaddata.s7 import make_s7_metadata_df, make_s7_df, load_s7_metadata_df, load_s7_df

    nthreads = 4
    eline_SNR_min = 3
    eline_ANR_min = 3

    # Create the DataFrames
    make_s7_metadata_df()
    make_s7_df(gals=["NGC1068", "MARK573", "NGC6860"],
                eline_SNR_min=eline_SNR_min,
                eline_ANR_min=eline_ANR_min,
                correct_extinction=True,
                metallicity_diagnostics=["N2Ha_PP04",],
                nthreads=nthreads)

Produces the following output:

File ~/python/Modules/spaxelsleuth/spaxelsleuth/loaddata/s7.py:665, in make_s7_df(eline_SNR_min, eline_ANR_min, correct_extinction, gals, sigma_gas_SNR_min, line_flux_SNR_cut, missing_fluxes_cut, line_amplitude_SNR_cut, flux_fraction_cut, sigma_gas_SNR_cut, vgrad_cut, metallicity_diagnostics, nthreads)
    663 logger.info(f"beginning pool...")
    664 pool = multiprocessing.Pool(min([nthreads, len(gals)]))
--> 665 res_list = np.array((pool.map(_process_s7, args_list)))
    666 pool.close()
    667 pool.join()

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (3, 3) + inhomogeneous part.

The error does not occur if nthreads = 1.