missing settings files

[mattcieslak]

Hi Okan,

I'm seeing a new error on some HBCD runs:

Stderr:
        terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
          what():  boost::filesystem::status: Function not implemented: "/src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.
json"
        Aborted (core dumped)

Are there files outside of bin that should be keeping for TORTOISE to work?

[eurotomania]

Hi Matt,

Are you running the TORTOISEProcess executable or just DRBUDDI?

Yes. For TORTOISEProcess, there are some settings files that need to be present. For DRBUDDI, no, it can run on its own.

On Mon, May 8, 2023 at 2:46 PM Matt Cieslak @.***> wrote:

Hi Okan,

I'm seeing a new error on some HBCD runs:

Stderr: terminate called after throwing an instance of 'boost::filesystem::filesystem_error' what(): boost::filesystem::status: Function not implemented: "/src/TORTOISEV4/bin/../settings/system_settings/default_system_settings. json" Aborted (core dumped)

Are there files outside of bin that should be keeping for TORTOISE to work?

— Reply to this email directly, view it on GitHub https://github.com/eurotomania/TORTOISEV4/issues/5, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFQZWEGI2TI7FA3D67K3QTXFE5SBANCNFSM6AAAAAAX2J256E . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

[mattcieslak]

This came from a DRBUDDI run, very strange

[eurotomania]

I checked the code and everything is fine. Yes really weird.

The code is under src/tools/DRBUDDI/DRBUDDI_main.cxx

lines 18-25.

As you can see, the file is checked for existence there. Will investigate more.

On Mon, May 8, 2023 at 3:10 PM Matt Cieslak @.***> wrote:

This came from a DRBUDDI run, very strange

— Reply to this email directly, view it on GitHub https://github.com/eurotomania/TORTOISEV4/issues/5#issuecomment-1538902284, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFQZWGUEFA6PCMVWUE26C3XFFAK3ANCNFSM6AAAAAAX2J256E . You are receiving this because you commented.Message ID: @.***>

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

[mattcieslak]

In the docker build I was only copying over the bin directory. I'm copying the settings directory into the image now too, maybe that will solve it

[eurotomania]

Also can you tell me your boost version?

On Mon, May 8, 2023 at 3:23 PM M. Okan Irfanoglu @.***> wrote:

I checked the code and everything is fine. Yes really weird.

The code is under src/tools/DRBUDDI/DRBUDDI_main.cxx

lines 18-25.

As you can see, the file is checked for existence there. Will investigate more.

On Mon, May 8, 2023 at 3:10 PM Matt Cieslak @.***> wrote:

This came from a DRBUDDI run, very strange

— Reply to this email directly, view it on GitHub https://github.com/eurotomania/TORTOISEV4/issues/5#issuecomment-1538902284, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFQZWGUEFA6PCMVWUE26C3XFFAK3ANCNFSM6AAAAAAX2J256E . You are receiving this because you commented.Message ID: @.***>

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

[eurotomania]

Sure that would solve it but it should not be necessary.

Maybe that boost version doesnt behave well under docker. I have had similar issues reported to me while running it under tmpfs (i.e. RAM drive) before with boost.

On Mon, May 8, 2023 at 3:25 PM M. Okan Irfanoglu @.***> wrote:

Also can you tell me your boost version?

On Mon, May 8, 2023 at 3:23 PM M. Okan Irfanoglu @.***> wrote:

I checked the code and everything is fine. Yes really weird.

The code is under src/tools/DRBUDDI/DRBUDDI_main.cxx

lines 18-25.

As you can see, the file is checked for existence there. Will investigate more.

On Mon, May 8, 2023 at 3:10 PM Matt Cieslak @.***> wrote:

This came from a DRBUDDI run, very strange

— Reply to this email directly, view it on GitHub https://github.com/eurotomania/TORTOISEV4/issues/5#issuecomment-1538902284, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFQZWGUEFA6PCMVWUE26C3XFFAK3ANCNFSM6AAAAAAX2J256E . You are receiving this because you commented.Message ID: @.***>

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

[mattcieslak]

Currently 1.76.0: https://github.com/PennLINC/qsiprep_build/blob/main/Dockerfile_TORTOISE#L28

[eurotomania]

Me too actually.

So I will test it under docker.

On Mon, May 8, 2023 at 3:27 PM Matt Cieslak @.***> wrote:

Currently 1.76.0: https://github.com/PennLINC/qsiprep_build/blob/main/Dockerfile_TORTOISE#L28

— Reply to this email directly, view it on GitHub https://github.com/eurotomania/TORTOISEV4/issues/5#issuecomment-1538925075, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFQZWBLLS4RLYFFR6S76ALXFFCKNANCNFSM6AAAAAAX2J256E . You are receiving this because you commented.Message ID: @.***>

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

[mattcieslak]

ok, this may be a false alarm. It looks like there was an unrelated issue with memory that was showing up as this sometimes :(

[mattcieslak]

The memory issue was unrelated, this unfortunately came up again.

background: being run on a centos7 machine via apptainer (rebranded singularity) from the pennbbl/qsiprep:unstable image that has the most recent TORTOISE installed (commmit 9fb5296649ef6de802fde25cca4cb2893a94c64d).

The actual error message is

terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
  what():  boost::filesystem::status: Function not implemented: "/src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.json"
command.txt: line 1: 151819 Aborted                 DRBUDDI -d /cbica/comp_space/abcd_qsiprep/job-4698856-sub-326427/.git/wkdir/qsiprep_wf/single_subject_326427_wf/dwi_preproc_ses_V02_run_1_wf/hmc_sdc_wf/drbuddi_sdc_wf/drbuddi/drbuddi_down_dwi.nii -u /cbica/comp_space/abcd_qsiprep/job-4698856-sub-326427/.git/wkdir/qsiprep_wf/single_subject_326427_wf/dwi_preproc_ses_V02_run_1_wf/hmc_sdc_wf/drbuddi_sdc_wf/drbuddi/drbuddi_up_dwi.nii --up_json /cbica/comp_space/abcd_qsiprep/job-4698856-sub-326427/.git/wkdir/qsiprep_wf/single_subject_326427_wf/dwi_preproc_ses_V02_run_1_wf/hmc_sdc_wf/drbuddi_sdc_wf/drbuddi/blip_up.json -s /cbica/comp_space/abcd_qsiprep/job-4698856-sub-326427/.git/wkdir/qsiprep_wf/single_subject_326427_wf/anat_preproc_wf/rigid_acpc_resample_unfatsat/sub-326427_ses-V02_run-1_T2w_lps_desaturated_trans.nii

I can see that the settings file is available in the container:

Apptainer> ls /src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.json
/src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.json
Apptainer> cat /src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.json
{
    "_comment": "Percentage of CPU cores to be used by TORTOISE as a float. Ex, 1: use all cores  0.1: use 10% of max cores",
    "PercentOfCpuCoresToUse": 1
}

I wonder if this could be related to https://github.com/boostorg/filesystem/issues/172, popping up also in https://github.com/PointCloudLibrary/pcl/issues/5197 and https://github.com/microsoft/WSL/issues/6994.

[eurotomania]

Yes it is probably related to these. I was reading them as well.

I will look into this more later but for now as a short term solution, I just changed that part of the code. All it was doing was to check the number of cores to use from that line. I made it a command line parameter for now.

Hopefully it should work. I just did a push.

However, the TORTOISEProcess executable uses it for more than that. So I will have to find a more permanent solution eventually.

On Tue, May 9, 2023 at 7:43 AM Matt Cieslak @.***> wrote:

The memory issue was unrelated, this unfortunately came up again.

background: being run on a centos7 machine via apptainer (rebranded singularity) from the pennbbl/qsiprep:unstable image that has the most recent TORTOISE installed (commmit 9fb5296 https://github.com/eurotomania/TORTOISEV4/commit/9fb5296649ef6de802fde25cca4cb2893a94c64d ).

The actual error message is

terminate called after throwing an instance of 'boost::filesystem::filesystem_error' what(): boost::filesystem::status: Function not implemented: "/src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.json" command.txt: line 1: 151819 Aborted DRBUDDI -d /cbica/comp_space/abcd_qsiprep/job-4698856-sub-326427/.git/wkdir/qsiprep_wf/single_subject_326427_wf/dwi_preproc_ses_V02_run_1_wf/hmc_sdc_wf/drbuddi_sdc_wf/drbuddi/drbuddi_down_dwi.nii -u /cbica/comp_space/abcd_qsiprep/job-4698856-sub-326427/.git/wkdir/qsiprep_wf/single_subject_326427_wf/dwi_preproc_ses_V02_run_1_wf/hmc_sdc_wf/drbuddi_sdc_wf/drbuddi/drbuddi_up_dwi.nii --up_json /cbica/comp_space/abcd_qsiprep/job-4698856-sub-326427/.git/wkdir/qsiprep_wf/single_subject_326427_wf/dwi_preproc_ses_V02_run_1_wf/hmc_sdc_wf/drbuddi_sdc_wf/drbuddi/blip_up.json -s /cbica/comp_space/abcd_qsiprep/job-4698856-sub-326427/.git/wkdir/qsiprep_wf/single_subject_326427_wf/anat_preproc_wf/rigid_acpc_resample_unfatsat/sub-326427_ses-V02_run-1_T2w_lps_desaturated_trans.nii

I can see that the settings file is available in the container:

Apptainer> ls /src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.json /src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.json Apptainer> cat /src/TORTOISEV4/bin/../settings/system_settings/default_system_settings.json { "_comment": "Percentage of CPU cores to be used by TORTOISE as a float. Ex, 1: use all cores 0.1: use 10% of max cores", "PercentOfCpuCoresToUse": 1 }

I wonder if this could be related to PointCloudLibrary/pcl#5197 https://github.com/PointCloudLibrary/pcl/issues/5197 or microsoft/WSL#6994 https://github.com/microsoft/WSL/issues/6994.

— Reply to this email directly, view it on GitHub https://github.com/eurotomania/TORTOISEV4/issues/5#issuecomment-1540006700, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFQZWAZN7RDL3JBBSDAVT3XFIUVXANCNFSM6AAAAAAX2J256E . You are receiving this because you commented.Message ID: @.***>

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

[mattcieslak]

I'm currently trying out using boost 1.77, do you think that could work? https://app.circleci.com/pipelines/github/PennLINC/qsiprep_build/145/workflows/b022dcbd-f7fc-43a4-972a-0b742c2123fa/jobs/1087

[eurotomania]

Not sure.

Let me know.

On Tue, May 9, 2023 at 9:58 AM Matt Cieslak @.***> wrote:

I'm currently trying out using boost 1.77, do you think that could work? https://app.circleci.com/pipelines/github/PennLINC/qsiprep_build/145/workflows/b022dcbd-f7fc-43a4-972a-0b742c2123fa/jobs/1087

— Reply to this email directly, view it on GitHub https://github.com/eurotomania/TORTOISEV4/issues/5#issuecomment-1540181126, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFQZWCGOD4DTAOOWVDNUHDXFJERRANCNFSM6AAAAAAX2J256E . You are receiving this because you commented.Message ID: @.***>

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University

[mattcieslak]

boost 1.77 fixed it!

[eurotomania]

Woow.

Great to know. Thank you Matt.

On Tue, May 9, 2023 at 3:07 PM Matt Cieslak @.***> wrote:

boost 1.77 fixed it!

— Reply to this email directly, view it on GitHub https://github.com/eurotomania/TORTOISEV4/issues/5#issuecomment-1540746957, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFQZWHFVFOSSWDGKBKTKOTXFKIXHANCNFSM6AAAAAAX2J256E . You are receiving this because you commented.Message ID: @.***>

-- M. Okan Irfanoglu Graduate Student & GTA Computer Sciences & Engineering Ohio State University