justicenode / files_external_ipfs

Nextcloud App to add ipfs external storage support
GNU Affero General Public License v3.0
67 stars 9 forks source link

Files = 0kb ? #4

Open Attil69 opened 4 years ago

Attil69 commented 4 years ago

Hi, and thanks for this contribution, having IPFS on nextcloud is just a great idea ! I installed V0.2 of your app. Everything seemed to work fine : the files appeared, but then I noticed they were all 0KB... When I tried to read or download one, it didn't work.

It works if I do that in a terminal with ipfs commands, but not using nextcloud interface...

Any Idea of what I could do to correct that problem ?

justicenode commented 4 years ago

First what kind of ipfs node do you have, and what version? and if you add the files with ipfs directly do they show up in nextcloud? Gonna need a little bit more information to help you

Attil69 commented 4 years ago

Hi and thank you for such celerity :-) I think i made an ipfs alone install : ipfs version 0.5.1 If I upload an ipfs file with nextcloud, it shows up but isn't correct (0kb and not downloadable) If i upload with cp command in terminal, it's ok and I can upload & get the files... Weird.

Attil69 commented 4 years ago

Can I ask you where are located the error logs of your apps ? and what datas are written in database ?

justicenode commented 4 years ago

Theres no need to write anything to the db. Error logs i haven't figured out how to do them nicely. Im guessing youre using the go-ipfs implementation then. that one does not yet support metadata which is kinda required for this app to work properly. See https://github.com/ipfs/go-ipfs/issues/6920

Yeah i think thats pretty much all i can do for you, sadly

Attil69 commented 4 years ago

Bingo. Nevermind, I'll upload with terminal. A bit borring, but nothing too mean, it's allright.

Thank you very much for replying so quickly, your app is great ;-)

zicmama commented 4 years ago

same problem for me... I can access with "ipfs files" from CLI, but not from NC. Is a the go-ipfs 0.5.1 upgrade problem ??

justicenode commented 4 years ago

probably you should be able to see the files though. if you add them to the same directory as you set in nextcloud. go-ipfs doesnt have timestamps yet. so as a workaround i just use the current time but nextcloud doesnt really like that but reading should work

TitaniumCoder477 commented 4 years ago

I have the same problem with 0 KB files upon upload.

WIL-IPFS-SRV / 192.168.1.43

To                         Action      From
--                         ------      ----
192.168.1.43               ALLOW       192.168.1.0/24            
192.168.1.43 4001/tcp      ALLOW       Anywhere                  
192.168.1.43 8080/tcp      ALLOW       Anywhere                  
192.168.1.43 5001/tcp      ALLOW       192.168.1.0/24            

192.168.1.1                ALLOW OUT   192.168.1.43              
192.168.1.0/24             DENY OUT    192.168.1.43

WIL-NEXTCLOUD-SRV / 192.168.1.21

To                         Action      From
--                         ------      ----
192.168.1.21               ALLOW IN    192.168.1.0/24            
192.168.1.21 443/tcp       ALLOW IN    Anywhere                  
192.168.1.21 9981          ALLOW IN    Anywhere                  
192.168.1.21 9982          ALLOW IN    Anywhere                  

192.168.1.1                ALLOW OUT   192.168.1.21              
192.168.1.43 5001/tcp      ALLOW OUT   192.168.1.21              
192.168.1.131 9000/tcp     ALLOW OUT   192.168.1.21              
192.168.1.0/24             DENY OUT    192.168.1.21         

Symptom:

  1. I can upload files just fine using http://192.168.1.43:5001/webui
  2. I can download these same files using the Nextcloud web UI
  3. I can also delete the files using the Nextcloud web UI
  4. But when I try to upload a file using the Nextcloud web UI, the file appears to upload (size and speed on progress bar), but after the progress bar goes away and file appears, it is just 0 KB -- aka, an empty file
  5. I can download the file from either IPFS web UI or Nextcloud web UI, but the file is empty

Screenshot showing progress bar: https://ipfs.io/ipfs/QmTRCtsQrzx6fPFV1twPdDzJvvjBELLHYP16GqxGooPuwz Screenshot showing 0 KB: https://ipfs.io/ipfs/QmSe3Dsrci4marzz2pSGaDCYBDe2ws9NBMF7vqWQupi2R2

UPDATE: I just now noticed in the IPFS web UI that it shows as "go-ipfs v0.5.0". What version of IPFS do I need to make this work?

justicenode commented 4 years ago

go-ipfs does not yet support metadata. See https://github.com/ipfs/go-ipfs/issues/6920. Currently you can only really get it to work with the js-ipfs implementation. I'll test and fix bugs once go-ipfs has metadata. Unless you find a bug using js-ipfs

h0cin commented 4 years ago

The same issue happens with ipfs v0.6.0, one thing to note is the ipfs-node's message when uploading a file called hola.txt (capturing the tcp/5001 port stream):

POST /api/v0/files/write?arg=%2F ipfs-node%2Fhola. txt. ocTrans ferId1610694374. part&create=true&truncate=true HTTP/1.1  
Host: 127.0.0.1:5001                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
Accept: */*                                                                                                                                                                                                                                                                  

Content-Type: multipart/form-data; boundary=—-5f36efcc97100                                                                                                                                                                                                                                 
Content-Length: 244 

—-5f36efcc97100                                                                                                                                                                                                                                                               
Content-Disposition: form-data; name="file"; filename="ipfs-node/hola.txt.ocTransferId1610694374. part"                                                                                                                                                                      
Content-Type: application/octet-stream                                                                                                                                                                                                                                       

mode: 0644                                                                                                                                                                                                                                                                   
mtime: 1597435852                                                                                                                                                                                                                                                            
hola 

—-5f36efcc97100-—
HTTP/1.1 500 Internal Server Error
Access-Control-Allow-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Access-Control-Expose-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Content-Type: application/json
Server: go-ipfs/0.6.0
Trailer: X-Stream-Error
Vary: Origin
Date: Fri, 14 Aug 2020 20:16
Transfer-Encoding: chunked

This is the error:

{"Message":"nalformed MIME header: missing colon: %!q(MISSING) hola" ,"Code":0, "Type" :"error"}

After this, the upload seems to go through but file is uploaded as 0kb (I'm not aware if I'm using metadata or not) and the content is unreadable.

Thank you!

justicenode commented 4 years ago

@h0cin youre using go-ipfs/0.6.0 which currently does not have metadata. Which means writing won't work. If you really want to get this working you can use js-ipfs.

imopro commented 4 years ago

@h0cin youre using go-ipfs/0.6.0 which currently does not have metadata. Which means writing won't work. If you really want to get this working you can use js-ipfs.

hello, You can take a look at this article:

https://medium.com/pinata/how-to-include-metadata-with-your-ipfs-uploads-19813513320f

This author shows how to include metadata when uploading files.

I hope you can improve the IPFS app. It's a great idea to combine with IPFS.

Thank you for your contribution!