Missing proof upload

[selroc]

PRP Proofs are not uploaded to Primenet server.

[tdulcet]

Yes, the PrimeNet script does not yet fully support GpuOwl. See this thread for more information: https://www.mersenneforum.org/showthread.php?t=27264.

I actually just finished implementing PRP proof uploading yesterday and was able to successfully upload a proof file, so hopefully I will be able to finish testing it locally in the next couple of weeks, before pushing the changes so other users can help test it…

[selroc]

Yes, the PrimeNet script does not yet fully support GpuOwl. See this thread for more information: https://www.mersenneforum.org/showthread.php?t=27264.

I actually just finished implementing PRP proof uploading yesterday and was able to successfully upload a proof file, so hopefully I will be able to finish testing it locally in the next couple of weeks, before pushing the changes so other users can help test it…

You could check Preda's primenet.py script which does proof upload, see preda/gpuowl tools directory.

[tdulcet]

You could check Preda's primenet.py script which does proof upload, see preda/gpuowl tools directory.

Yes, I am aware of the script included with GpuOwl. Unfortunately, it does not meet the needs of our PrimeNet script, as it only supports Python 3.6+, it has very little error checking, lots of dead code and it requires reading the entire (possibly several GiB) proof file into memory. It will in some cases reupload parts of the proofs that have already been uploaded. It also archives all proof files to an uploaded directory after uploading them, with no way for the user to disable.

Anyway, as I said, I have already implemented PRP proof uploading for our PrimeNet script. I decided to directly port the Prime95/MPrime C code to Python, so our PrimeNet script will include identical error checking and also only require reading one 5 MiB chunk into memory at a time. I am just waiting until I have collected enough PRP proof files to be able to thoroughly test my implementation…

[selroc]

I have developed this script https://github.com/SEL-ROC/gpuowl-development/blob/master/copy_proofs.sh which copies proof files to a specified location. I use it with an NFS share to archive proof files. multiple source directories can be specified as follows:

copy_proofs '/home/x/work0 /home/x/work1 /home/x/work2 /home/x/work3' destination_directory

where destination_directory is an NFS share.

[tdulcet]

I have developed this script https://github.com/SEL-ROC/gpuowl-development/blob/master/copy_proofs.sh which copies proof files to a specified location.

Yes, I saw the PR created for that script: https://github.com/preda/gpuowl/pull/177. Are you using it to move your proofs from the uploaded directory to your NFS share? Is there a specific feature you are requesting that I add to the PrimeNet script for your use case?

To be clear, our PrimeNet script will automatically delete the proof files after they are uploaded. There will also be an optional --archive_proofs option to specify a directory to archive the proofs after upload instead:

  --archive_proofs=PROOFARCHIVEDIR
                        Directory to archive PRP proof files after upload,
                        Default: none

Thus users could get the same effect as the other script by adding the --archive_proofs 'uploaded' option the first time they run our PrimeNet script.

[selroc]

I have developed this script https://github.com/SEL-ROC/gpuowl-development/blob/master/copy_proofs.sh which copies proof files to a specified location.

Yes, I saw the PR created for that script: preda/gpuowl#177. Are you using it to move your proofs from the uploaded directory to your NFT share? Is there a specific feature you are requesting that I add to the PrimeNet script for your use case?

To be clear, our PrimeNet script will automatically delete the proof files after they are uploaded. There will also be an optional --archive_proofs option to specify a directory to archive the proofs after upload instead:

  --archive_proofs=PROOFARCHIVEDIR
                        Directory to archive PRP proof files after upload,
                        Default: none

Thus users could get the same effect as the other script by adding the --archive_proofs 'uploaded' option the first time they run our PrimeNet script.

There's no special feature to request as the PROOFARCHIVEDIR can be a NFS share.

[tdulcet]

@selroc - PRP proof uploading support was added in https://github.com/tdulcet/Distributed-Computing-Scripts/commit/e7654dfb39682baaa0df1601d2cf0ccdb322182a. Feedback is welcome.

[selroc]

@selroc - PRP proof uploading support was added in e7654df. Feedback is welcome.

Both --upload_proofs and --archive_proofs will exit after job and not stay in the loop.

[tdulcet]

Both --upload_proofs and --archive_proofs will exit after job and not stay in the loop.

Yes, the --upload_proofs option is designed to exit after uploading the proofs:

  --upload_proofs       Report assignment results, upload all PRP proofs and
                        exit.

The --archive_proofs option does nothing except update the config file with the directory to archive the PRP proof files (see usage in comment above). This option only needs to be provided once, as the value is saved into the config file. Maybe I should change it to --archive_dir to make this clearer. The script should not exit after this option, unless you do not provide the mandatory argument, in which case you would get a usage error.

Anyway, you do not need to use either of these options, as it will automatically upload any PRP proofs, by default every hour. I would recommend always adding the -d/--debug option, so you can see all of the output.