search

NEFSC / READ-SSB-CHAJI-Effort-Displacement---Scallop

Other
0 stars 0 forks source link

replication test #147

Closed mle2718 closed 2 years ago

mle2718 commented 2 years ago

run replication test Switch to main and pull. Run the data extracting, data processing, combine_cable_routes, and scallop_analysis.Rmds

The scallop_analysis.Rmd will open a dialog box will pop up to select the directory to store the FishSETFolder. It's up to you where to put it. I put it inside the project folder. I also added it to .gitignore, so as long as you're on main, you should not have to worry about pushing the project datasets to github accidentally.

mle2718 commented 2 years ago

i got 1/2 way through our replication test. I think I fixed the combine_cable_routes bug.

mchaji commented 2 years ago

Ran replication test with no errors or issues in data extracting, data processing, and combine_cable_routes rmds.

I initially got the following error in the scallop_analysis rmd:

Error in select(., -DOLLAR, -POUNDS, -LANDED, -DOLLAR_ALL_SP_2020) : 
  unused arguments (-DOLLAR, -POUNDS, -LANDED, -DOLLAR_ALL_SP_2020)

which I fixed by checking dplyr 1.0.9 and rerunning the code chunk, which fixed the error above but created a new one Error in ungroup(.) : could not find function "ungroup"

After unchecking dplyr 1.0.9 & dplyr 1.0.8 and rechecking dplyr 1.0.8 by itself the code chunk ran with no errors - go figure

Everything else runs, but I got a few warning messages. I'm not sure if I should be worried about these, though...

  1. Error in eval(ele) : object 'params' not found ( a few instances of this are found throughout)

  2. TRIPID OPERATOR OPERNUM NSUBTRIP CREW VTR_PORTNUM IMGID YEAR DATE_TRIP VTR_PORT VTR_STATE TRIP_LENGTH PERMIT.y DEALNUM DOLLAR_ALL_SP GEARCODE SECGEARFISH SPPNAME geoid namelsad state_fips port_lat port_lon previous_namelsad previous_state_fips previous_geoid previous_port_lat previous_port_lon Date Time scallop_fishing_year TRIP_ID Plan Code Program Code Area Identifier ftpt GC LA hours DB_LANDING_YEAR TRIP_COST_2020_DOL TRIP_COST_WINSOR_2020_DOL OBSERVED_COST_DUMMY DDLAT DDLON MN30SQID MN10SQID NAME Zone_ID Area_ID2 fleet fleetAssignPlaceholder GEARCODE_OLD DOLLAR POUNDS LANDED DOLLAR_ALL_SP_2020 OPERATING_PROFIT_2020 dup_trip is empty. Consider removing from the data set.

mle2718 commented 2 years ago

which I fixed by checking dplyr 1.0.9 and rerunning the code chunk, which fixed the error above but created a new one Error in ungroup(.) : could not find function "ungroup"

After unchecking dplyr 1.0.9 & dplyr 1.0.8 and rechecking dplyr 1.0.8 by itself the code chunk ran with no errors - go figure

This is weird. Super weird.

Everything else runs, but I got a few warning messages. I'm not sure if I should be worried about these, though...

Are you running chunk by chunk, doing knit to html? I've noticed the eval(ele) issue when I run chunk-by-chunk in other projects, but it goes away when I knit all the way to html.

TRIPID OPERATOR OPERNUM NSUBTRIP CREW VTR_PORTNUM IMGID YEAR DATE_TRIP VTR_PORT VTR_STATE TRIP_LENGTH PERMIT.y DEALNUM DOLLAR_ALL_SP GEARCODE SECGEARFISH SPPNAME geoid namelsad state_fips port_lat port_lon previous_namelsad previous_state_fips previous_geoid previous_port_lat previous_port_lon Date Time scallop_fishing_year TRIP_ID Plan Code Program Code Area Identifier ftpt GC LA hours DB_LANDING_YEAR TRIP_COST_2020_DOL TRIP_COST_WINSOR_2020_DOL OBSERVED_COST_DUMMY DDLAT DDLON MN30SQID MN10SQID NAME Zone_ID Area_ID2 fleet fleetAssignPlaceholder GEARCODE_OLD DOLLAR POUNDS LANDED DOLLAR_ALL_SP_2020 OPERATING_PROFIT_2020 dup_trip is empty. Consider removing from the data set.

That's troubling. Does the scallop analysis 'match' the one that I did? Can you show me where this warning is?

mchaji commented 2 years ago

which I fixed by checking dplyr 1.0.9 and rerunning the code chunk, which fixed the error above but created a new one Error in ungroup(.) : could not find function "ungroup"

After unchecking dplyr 1.0.9 & dplyr 1.0.8 and rechecking dplyr 1.0.8 by itself the code chunk ran with no errors - go figure

This is weird. Super weird.

Everything else runs, but I got a few warning messages. I'm not sure if I should be worried about these, though...

Are you running chunk by chunk, doing knit to html? I've noticed the eval(ele) issue when I run chunk-by-chunk in other projects, but it goes away when I knit all the way to html.

TRIPID OPERATOR OPERNUM NSUBTRIP CREW VTR_PORTNUM IMGID YEAR DATE_TRIP VTR_PORT VTR_STATE TRIP_LENGTH PERMIT.y DEALNUM DOLLAR_ALL_SP GEARCODE SECGEARFISH SPPNAME geoid namelsad state_fips port_lat port_lon previous_namelsad previous_state_fips previous_geoid previous_port_lat previous_port_lon Date Time scallop_fishing_year TRIP_ID Plan Code Program Code Area Identifier ftpt GC LA hours DB_LANDING_YEAR TRIP_COST_2020_DOL TRIP_COST_WINSOR_2020_DOL OBSERVED_COST_DUMMY DDLAT DDLON MN30SQID MN10SQID NAME Zone_ID Area_ID2 fleet fleetAssignPlaceholder GEARCODE_OLD DOLLAR POUNDS LANDED DOLLAR_ALL_SP_2020 OPERATING_PROFIT_2020 dup_trip is empty. Consider removing from the data set.

That's troubling. Does the scallop analysis 'match' the one that I did? Can you show me where this warning is?

line number 364 is what I think triggers it:

empty_vars_filter(scallop0322MainDataTable, project = project, remove = FALSE)

Empty vars check for scallop0322MainDataTable dataset on 20220825 
TRIPID OPERATOR OPERNUM NSUBTRIP CREW VTR_PORTNUM IMGID YEAR DATE_TRIP VTR_PORT VTR_STATE TRIP_LENGTH PERMIT.y DEALNUM DOLLAR_ALL_SP GEARCODE SECGEARFISH SPPNAME geoid namelsad state_fips port_lat port_lon previous_namelsad previous_state_fips previous_geoid previous_port_lat previous_port_lon Date Time scallop_fishing_year TRIP_ID Plan Code Program Code Area Identifier ftpt GC LA hours DB_LANDING_YEAR TRIP_COST_2020_DOL TRIP_COST_WINSOR_2020_DOL OBSERVED_COST_DUMMY DDLAT DDLON MN30SQID MN10SQID NAME Zone_ID Area_ID2 fleet fleetAssignPlaceholder GEARCODE_OLD DOLLAR POUNDS LANDED DOLLAR_ALL_SP_2020 OPERATING_PROFIT_2020 dup_trip is empty. Consider removing from the data set.
mchaji commented 2 years ago

I was running it chunk by chunk. I tried knitting to pdf but got an error on line 89 scallop analysis rmd knitting error pdf

Looks like it's fine when I knit to HTML but then I get the NAME error. scallop analysis rmd knitting error html

I think the HTML is my fault though because I deleted the most recent data set to test an issue I had with something else in another branch so I'm going to restart the extraction and processing then try to knit to HTML and let you know.

mchaji commented 2 years ago

I was running it chunk by chunk. I tried knitting to pdf but got an error on line 89 scallop analysis rmd knitting error pdf

Looks like it's fine when I knit to HTML but then I get the NAME error. scallop analysis rmd knitting error html

I think the HTML is my fault though because I deleted the most recent data set to test an issue I had with something else in another branch so I'm going to restart the extraction and processing then try to knit to HTML and let you know.

Actually, when I ran it chunk by chunk the error in 525 didn't come up - I just double-checked in my history. I'll try it from scratch again just in case but I thought that was strange

mle2718 commented 2 years ago

@mchaji the line 89 error corresponds to this code:

https://github.com/NEFSC/READ-SSB-CHAJI-Effort-Displacement---Scallop/blob/a8ea9a4b704747b739a669ce2385cf1f94e8b8eb/analysis_code/scallop_analysis_0322.Rmd#L80-L90

This uses a bit of FishSET code to find and erase the existing FishSET project. I think it's only a half error. The code should open up a dialog box for you to select the folder that contains the "FishSETFolder." But if you don't make a selection, you might get this error. I've been putting the FishSETFolder inside this project, so you probably just need to select "ok" or 'yes' on the dialog.

mle2718 commented 2 years ago

@mchaji was also able to run the code all the way through.