akvo / akvo-flow-services

Akvo Flow service applications for reporting, bulk uploads and others
Other
4 stars 1 forks source link

Bulk upload does not work #155

Closed valllllll2000 closed 4 years ago

valllllll2000 commented 7 years ago

Steps to reproduce: Fill out some surveys on mobile device in flight mode Once they are in exported state zip the data folder inside akvoflow, call it dataz.zip Pull dataz.zip on PC Using chrome browser I have tried to upload data.zip via the data/bulk upload data tab In messages I can see the file as processed. Here is the stacktrace of the issue: Caused by: java.lang.IllegalArgumentException: /data/files/58cffa26-3f9b-4479-b772-ef10ef9fc50d.zip is not a relative path at clojure.java.io$as_relative_path.invoke(io.clj:405) at clojure.java.io$file.invoke(io.clj:416) at clojure.lang.AFn.applyToHelper(AFn.java:156) at clojure.lang.RestFn.applyTo(RestFn.java:132) at clojure.core$apply.invoke(core.clj:632) at me.raynes.fs$file.doInvoke(fs.clj:56) at clojure.lang.RestFn.invoke(RestFn.java:423) at me.raynes.fs.compression$unzip$target_file__4141.invoke(compression.clj:18) at me.raynes.fs.compression$unzip.invoke(compression.clj:20) at akvo.flow_services.uploader$unzip_file.invoke(uploader.clj:80) at akvo.flow_services.uploader$bulk_upload.invoke(uploader.clj:258) at akvo.flow_services.scheduler.BulkUploadJob.execute(scheduler.clj:73) at org.quartz.core.JobRunShell.run(JobRunShell.java:213)

iperdomo commented 7 years ago

Once they are in exported state zip the data folder inside akvoflow, call it dataz.zip Pull dataz.zip on PC

you see that your zip file has an absolute path to /data/ which doesn't exist on the server.

valllllll2000 commented 7 years ago

After talking to @iperdomo he has found the issue: The zip file I am trying to upload has absolute paths, it was generated by an App I have on my device, called ASTRO:

 /data/                                                                                                                    
 /data/files/                                                                                                              
 /data/media/                                                                                                              
 /data/files/58cffa26-3f9b-4479-b772-ef10ef9fc50d.zip                                                                      
 /data/files/335b26c7-f26d-4a83-98c3-c1243972cb77.zip                                                                      
 /data/media/cf92b2a2-1a8a-4923-a8de-ea262a32b7dd.jpg                                                                      
 /data/media/631be731-6b21-4d10-bff4-7f7ce8ab1ef5.jpg                                                                      
 /data/media/b4369ac6-400d-4ab1-9e9a-9c18a4ea09f9.mp4                                                                      

A zip file generated by the PC has the following structure:

data/
data/files/
data/files/1a197398-02f1-4b28-961a-a50793c16c1c.zip
data/media/
data/media/22c15474-d6bf-4a5b-94a6-8a8b21fb7f3e.jpg
data/media/66c1f615-3fde-4338-950d-fd14df5c6868.jpg

In my opinion flow-services should be able to read both files but if it si decided that this is not a bug, we should explain in the testing plan about this and inform the users that the zip file has to be generated by a computer and not any mobile app.

muloem commented 7 years ago

@valllllll2000 I think it shouldn't matter where the zip file is generated so I think we can keep this recorded as a bug and fix it. 👍

iperdomo commented 7 years ago

@valllllll2000 we do inform the users to use a computer zip program - http://flowsupport.akvo.org/article/show/bulk-upload-data-tab

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.