Uploader does not upload

[behinger]

I tried using the upload to share a dataset with subfolders.

While the tool connects to server & detects what is there and what needs to be uploaded, it simply does not upload things.

This is the log (API removed)

java -jar ~/Downloads/DVUploader-v1.1.0.jar -server=https://darus.uni-stuttgart.de -key=XXX -did=doi:10.18419/darus-2029 -recurse -listonly *            

----------------------------------------------------------------------------------                                                                                                          

TTTTT  DDD   L    Texas                                                                                                                                                                     
  T    D  D  L    Digital                                                                                                                                                                   
  T    DDD   LLL  Library                                                                                                                                                                   

DVUploader - a command-line application to upload files to any Dataverse Dataset                                                                                                            
Developed for the Dataverse Community                                                                                                                                                       

----------------------------------------------------------------------------------                                                                                                          

***Parsing arguments:***                                                                                                                                                                    

Using server: https://darus.uni-stuttgart.de                                                                                                                                                
Using apiKey: XXX                                                                                                                                        
Adding content to: doi:10.18419/darus-2029                                                                                                                                                  
Will recurse into subdirectories                                                                                                                                                            
List Only Mode   

Request to upload: 
DVUploaderLog__1623759256268.log                                                                                                                               [544/1652]
Request to upload: DVUploaderLog__1623759276518.log                                                                                                                                         
Request to upload: DVUploaderLog__1623759284812.log                                                                                                                                         
Request to upload: participants.json                                                                                                                                                        
Request to upload: participants.json~                                                                                                                                                       
Request to upload: participants.tsv                                                                                                                                                         
Request to upload: participants.tsv~                                                                                                                                                        
Request to upload: README                                                                                                                                                                   
Request to upload: sub-01                                                                                                                                                                   
Request to upload: sub-02                                                                                                                                                                   

***Starting to Process Upload Requests:***                                                                                                                                                  

PROCESSING(F): dataset_description.json                                                                                                                                                     
               Does not yet exist on server.                                                                                                                                                

PROCESSING(F): DVUploaderLog__1623759256268.log                                                                                                                                             
               Does not yet exist on server.                                                                                                                                                

PROCESSING(F): DVUploaderLog__1623759276518.log                                                                                                                                             
               Does not yet exist on server.                                                                                                                                                

PROCESSING(F): DVUploaderLog__1623759284812.log                                                                                                                                             
               Does not yet exist on server.                                                                                                                                                

PROCESSING(F): participants.json                                                                                                                                                            
               Does not yet exist on server.                                                                                                                                                

PROCESSING(F): participants.json~                                                                                                                                                           
               Does not yet exist on server.      

...

PROCESSING(F): sub-02/eeg/sub-02_task-FlankerBackwardWalking_run-02_eeg.json                                                                                                                
               Does not yet exist on server.   

***Execution Complete.***                                                                                                                                                                   

(Cut out some files - by now I uploaded it using the zip trick, but it is annoying and I prefer something else).

[qqmyers]

With the -listonly flag, DVUploader only tells you what it would do, but does not proceed with uploads.

Also - FWIW: v1.1.0 won't do direct S3 uploads against Dataverse v4.20 - you can use DVUploader v1.0.9 for that. Otherwise (and I see you don't use the -directupload flag) I think v1.1.0 should still be compatible with Dataverse 4.20.

[behinger]

ah oops, sorry that was a mistake in copying it out. I of course tested it without the flag.

Regarding the FWIW: not sure what that means - how do I figure out my DataVerse Server Version? And should I try the v1.0.9 or -directupload flag?

[qqmyers]

The version is shown at the bottom right - https://darus.uni-stuttgart.de shows v4.20. (There's also an api /api/info/version I think)

To debug what's going wrong - can you send the log file from the run without the -listonly flag? DVUploader currently writes most things to the console and log file, but the log file is more complete.

[behinger]

I'm confused, because now it seems to work, at least a single file I successfully uploaded.

[behinger]

ok. on a new dataset on the server: Now I get an error back - I did upload some files successfully without the "recurse" flag.

Logfile is a good hint, because here I find:

java.lang.StringIndexOutOfBoundsException: begin 1, end 0, length 33
        at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319)
        at java.base/java.lang.String.substring(String.java:1874)
        at org.sead.uploader.dataverse.DVUploader.uploadDatafile(DVUploader.java:431)
        at org.sead.uploader.AbstractUploader.uploadDatafile(AbstractUploader.java:454)
        at org.sead.uploader.AbstractUploader.processRequests(AbstractUploader.java:205)
        at org.sead.uploader.dataverse.DVUploader.main(DVUploader.java:120)

DVUploaderLog__1623761485852.log

[qqmyers]

Hmm - it may be having trouble with * as the path. When Dataverse added support for tracking folder paths, logic was added to DVUploader to map the local path to a folder path. The substring call that's failing is part of that. Testing was probably always done against a subfolder (e.g. with 'test' on the command line to get files in the ./test directory relative to where the uploader is being called from). If that's the case (can you confirm?) then it's a bug that we can track here.

[behinger]

nice! that's exactly it. (cd .. & * -> folder_name worked)