search

qri-io / qri

you're invited to a data party!
https://qri.io
GNU General Public License v3.0
1.1k stars 66 forks source link

qri save doesn't save body and readme #1267

Open hbruch opened 4 years ago

hbruch commented 4 years ago

Subsequent qri save calls don't save body.csv/readme.md:

$ qri version
0.9.8-dev

$ qri save -m "Try 0.9.8-dev to get body and readme saved"
for linked dataset [mfdz/nvbw_haltestellen]

dataset saved: mfdz/nvbw_haltestellen@/ipfs/QmZe611KfNwFy31Rn6RbMAYngwyBTohbfS2hfzyHjcc6DQ

$ qri status
for linked dataset [mfdz/nvbw_haltestellen]

  modified: readme (source: readme.md)
  modified: body (source: body.csv)

run `qri save` to commit this dataset
$ qri save -m "Try again 0.9.8-dev to get body and readme saved"
for linked dataset [mfdz/nvbw_haltestellen]

dataset saved: mfdz/nvbw_haltestellen@/ipfs/QmT75Dm6qufiVV6dPtSpWt2Brrev4HJPeveZawrNu81cSS
$ qri status
for linked dataset [mfdz/nvbw_haltestellen]

  modified: readme (source: readme.md)
  modified: body (source: body.csv)

run `qri save` to commit this dataset

qri version is commit 83b885020f7968

dustmop commented 4 years ago

Can't reproduce this, the command works correctly on my machine. Could you provide more information? Any of the following could help:

hbruch commented 4 years ago

I did not record the step by step creation of the dataset, however, I used different versions of qri CLI /qri desktop since dataset creation. Starting with precompiled qri 0.3.7, later 0.4, then different HEAD versions from github.

I attached the output of the smaller files below, for qri diff just the first lines. The last couple of commits, I did not change body at all.

Output qri diff:

$ qri diff
for linked dataset [mfdz/nvbw_haltestellen]

-3518647 elements. 74521 inserts. 2 deletes.

 body:
   0:
    +0: "Alb-Donau-Kreis;Allmendingen;Allmendingen;Schwimmbadweg;Allmendingen Schwimmbadweg;;de:08425:2999;normal;01.10.78;24.12.00;9.7173159;48.3301414;;;;Hst. ohne Bereich;;Zugang_und_OeV;;;;;Alle Richtungen;de:08425:2999:0:1;;;9.7173422;48.3300963;X;X;X;;;;;;;Schwimmbadweg;;;;;;;;;;;;;;;;;;;"
   1:
    +0: "Alb-Donau-Kreis;Allmendingen;Allmendingen;Lichseweg;Allmendingen Lichseweg;;de:08425:3000;normal;01.10.78;24.12.00;9.714882;48.3268021;;;;Hst. ohne Bereich;;Zugang_und_OeV;;;;;Alle Richtungen;de:08425:3000:0:1;;;9.7149364;48.3268288;X;X;X;;;;;;;Lichseweg;;;;;;;;;;;;;;;;;;;"
   2:
...

qri_log.txt qri_get_QmT75Dm6quf.txt qri_get_QmZe611KfNw.txt qri_get.txt

dustmop commented 4 years ago

Ok, interesting, I think I may have a reproduction here. It seems like readme difference checking is acting incorrectly. Just to make sure, you're not using transforms for this dataset, right?

Could you try this? 1) Make a backup of your readme.md, then delete it. 2) Run qri save, should make a commit because you've removed the readme. 3) Run qri save again, the hope is that now it won't make a commit and you'll get the message "error saving: no changes".

hbruch commented 4 years ago

Unexpectedly, already after renaming readme.md, saving resulted in "error saving: no changes":

$ mv readme.md readme.bak
$ qri save -m "Delete readme"
for linked dataset [mfdz/nvbw_haltestellen]

error saving: no changes
dustmop commented 4 years ago

Ok, that also makes sense. I think I know what the issue is. The previous version's readme is not being read correctly, so the differ thinks that it used to be an empty file. That's why the qri log file you sent is always saying "added readme.scriptBytes" for most of the commits. Will work on a fix, hope to have it in early next week.

dustmop commented 4 years ago

The readme problem should be fixed now, thanks to https://github.com/qri-io/qri/pull/1293. However, the fact that you're seeing body being dirty is strange. If you pull and rebuild, what happens?

b5 commented 4 years ago

hey @hbruch, did we ever find the proper solution to this? have you tried with the latest version of qri (v0.9.10)?