guilherber commented 9 months ago

After I changed my weather database, the algorithm finds problems to continue the simulation. I just changed it from csv to a json API, the data is the same and the Weather function works normally. I'm using version 2.1.3. I encounter errors during the DSSAT run.

Traceback (most recent call last): File "fortranformat_output.py", line 595, in _compose_i_string val = int(val) ValueError: invalid literal for int() with base 10: '375.18'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 39, in thread = int(future.result()['Profile'].str.extract('(\d+)').iloc[0].iloc[0]) - 1 File "lib\concurrent\futures_base.py", line 437, in result return self.get_result() File "lib\concurrent\futures_base.py", line 389, in get_result raise self._exception File "ib\concurrent\futures\thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "\main.py", line 15, in run_dssat dssat_env.run(crop=crop, weather=dados_clima(), management=man, soil=soil_profile) File "\run.py", line 216, in run management.write(filename=management_filename) File "management.py", line 325, in write outstr += section_obj.write() + '\n' File "sections.py", line 544, in write outstr += rowbased_write( File "\sections.py", line 351, in rowbased_write return writer.write(fields) File "FortranRecordWriter.py", line 42, in write return _output(self._eds, self._rev_eds, values) File "output.py", line 98, in output sub_string = _compose_i_string( File "fortranformat_output.py", line 597, in _compose_i_string raise ValueError("cannot convert '%s' to a integer" % str(val)) ValueError: cannot convert '375.18' to a integer

Maybe the problem is during the formation of topics, even changing the condition, the error follows...

def run_dssat(thread, crop, man, soil_profile, dssat_envs): dssat_env = DSSAT() dssat_env.setup() dssat_env.run(crop=crop, weather=dados_clima(), management=man, soil=soil_profile)

plant_gro_results = dssat_env.output.get('PlantGro', {})
df = pd.DataFrame(plant_gro_results)
df['Profile'] = f'Soil Profile {thread + 1}'
df['GWAD'] = df['GWAD'].round().astype(int)

# Add DSSAT instance to a list
dssat_envs[thread] = dssat_env

return df

Number of threads equal to soil_profiles number

threads = len(soil_profiles)

List to store results

output_dataframes = [None] * threads # Initialize with None for each thread

List to store DSSAT instances

dssat_envs = [None] * threads

with concurrent.futures.ThreadPoolExecutor(max_workers=threads) as executor: futures = [executor.submit(run_dssat, thread, crop, deepcopy(man), soil_profile, dssat_envs) for thread, soil_profile in enumerate(soil_profiles)]

for future in concurrent.futures.as_completed(futures):
    # Extract the thread index from the 'Profile' column
    thread = int(future.result()['Profile'].str.extract('(\d+)').iloc[0].iloc[0]) - 1
    output_dataframes[thread] = future.result()

daquinterop commented 9 months ago

By reading the Traceback I think there might be a type problem with your new data. Probably that API returns numeric values represented as strings. Make sure that all the fields in the dataframe have a numeric type.

guilherber commented 9 months ago

After I have solved the problem with the numerical data, I now encounter problem in my output. How do I ensure that I have all the necessary files in %TMP% to be able to continue the simulation?

Traceback (most recent call last): File "main.py", line 41, in thread = int(future.result()['Profile'].str.extract('(\d+)').iloc[0].iloc[0]) - 1 File "\lib\concurrent\futures_base.py", line 437, in result return self.get_result() File "\lib\concurrent\futures_base.py", line 389, in get_result raise self._exception File "\lib\concurrent\futures\thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "\main.py", line 15, in run_dssat dssat_env.run(crop=crop, weather=dados_clima(), management=man, soil=soil_profile) File "\run.py", line 270, in run assert f'{file}.OUT' in OUTPUT_FILES, \ AssertionError: Weather.OUT does not exist in Temp\dssatnihmuhyf

Process finished with exit code 1

daquinterop commented 9 months ago

It is a problem with the output. The model is running but it fails when fetching that specific file. What crop are you running?

guilherber commented 9 months ago

I'm using crop = Crop('soybean', '990004')

daquinterop commented 9 months ago

Just tested it and I did not find anything wrong with it. Can you zip the content of the DSSAT run path (Temp\dssatnihmuhyf) and attach it here? that would really help to catch the problem.

guilherber commented 9 months ago

dssatnihmuhyf.zip

daquinterop commented 9 months ago

Your Weather data seems to be incomplete. In 2022 it jumps from day 268 to day 296. Also, it seems that your solar radiation values are in the wrong units. The units must be MJ/m2-d. Thanks for reporting this, I'll add some extra checks to the Weather routines to make sure no data is missing in the input dataframe.

guilherber commented 9 months ago

Thank you for your reply, it was effective. Despite the corrections, I still have a zero simulation as output, can you recommend changes for me to reach variable values? Thank you!

RUN TRT FLO MAT TOPWT HARWT RAIN TIRR CET PESW TNUP TNLF TSON TSOC dap dap kg/ha kg/ha mm mm mm mm kg/ha kg/ha kg/ha t/ha 1 SB 1 26 88 1 0 -99 0 -99 -99 0 -99 0 217

RUN TRT FLO MAT TOPWT HARWT RAIN TIRR CET PESW TNUP TNLF TSON TSOC dap dap kg/ha kg/ha mm mm mm mm kg/ha kg/ha kg/ha t/ha

1 SB 1 26 88 1 0 -99 0 -99 -99 0 -99 0 217 RUN TRT FLO MAT TOPWT HARWT RAIN TIRR CET PESW TNUP TNLF TSON TSOC dap dap kg/ha kg/ha mm mm mm mm kg/ha kg/ha kg/ha t/ha 1 SB 1 26 88 1 0 -99 0 -99 -99 0 -99 0 217

RUN TRT FLO MAT TOPWT HARWT RAIN TIRR CET PESW TNUP TNLF TSON TSOC dap dap kg/ha kg/ha mm mm mm mm kg/ha kg/ha kg/ha t/ha 1 SB 1 26 88 1 0 -99 0 -99 -99 0 -99 0 217

RUN TRT FLO MAT TOPWT HARWT RAIN TIRR CET PESW TNUP TNLF TSON TSOC dap dap kg/ha kg/ha mm mm mm mm kg/ha kg/ha kg/ha t/ha 1 SB 1 26 88 1 0 -99 0 -99 -99 0 -99 0 217 0 1 2 3 4 5 6 7 8 9 10 Profile_Index
0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 ... .. .. .. .. .. .. .. .. .. .. 130 0 0 0 0 0 0 0 0 0 0 131 0 0 0 0 0 0 0 0 0 0 132 0 0 0 0 0 0 0 0 0 0 133 0 0 0 0 0 0 0 0 0 0 134 0 0 0 0 0 0 0 0 0 0

[135 rows x 10 columns]

Process finished with exit code 0

daquinterop / Py_DSSATTools

invalid literal for int() with base 10 issue #27

Number of threads equal to soil_profiles number

List to store results

List to store DSSAT instances