subutai-io / cdn

Gorjun is a golang replacement for Kurjun project.
Apache License 2.0
19 stars 13 forks source link

Same template of the same version should be overwritten on subsequent upload #220

Closed dilshat closed 6 years ago

dilshat commented 6 years ago

Currently if user uploads a template of the same name and version, two identically named templates are listed under the same owner (unless they have the same hash sum) The workflow should be like this: Check number of owners of the template a) Template has one owner - replace the template file with the new one b) Template has multiple owners - remove the uploading user from owner list, save the new template and include the user into its owner list. Thus we do not modify the other owner's older version of the template.
The uploading (and any other) user will always have only one template file per one version of the same name.

lbthomsen commented 6 years ago

I actually think it's the version parsing that's the issue - not the duplicated version. I just uploaded ubuntu-xenial@0.1.11 and https://mastercdn.subut.ai:8338/kurjun/rest/template/info?name=ubuntu-xenial&token= still reports:

[{"id":"136d90bf-83c7-48b9-89ad-44e8a2338834","hash":{"md5":"ceccb9b88af1f6b29779f86dcbc2be67","sha256":"6279a5672755928e479fa6ab3951bd3cecb661a5d0bb70d58c3837d818411720"},"size":82929586,"upload-date-formatted":"2018-01-31T07:31:08.879153746Z","upload-date-timestamp":"1517383868","name":"ubuntu-xenial","owner":["f9d6a36b3e9b1d985008e482680f2ddf584d98bd"],"parent":"ubuntu-xenial","version":"0.1.9","filename":"ubuntu-xenial-subutai-template_0.1.9_amd64.tar.gz","prefsize":"tiny","signature":{"f9d6a36b3e9b1d985008e482680f2ddf584d98bd":"\n-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA256\n\n136d90bf-83c7-48b9-89ad-44e8a2338834\n-----BEGIN PGP SIGNATURE-----\nVersion: Subutai v4.1.0\nComment: https://subutai.io/\n\nwsFcBAEBCAAQBQJacXC/CRBoDy3fWE2YvQAA9pUP/jsjkqGxGLBsh1QkoX5R\np1XJRmXO6bBfVhAd1SwAQ7uLdR1l6uRBFuSw4tL3vzb3nYGArI9CWKAHupve\nP0XlXHLOTou1Sy6e99osJOdDy060u2ToFCJ2JRbosxTCI6X+6CZ66YAnR37e\no0MSAP+wFO74mZTJDP7DVcSgJtvKZZbXR0Xv+T3Ze0SvInvAzpWtTv7vlgkP\n5eLjRgltsdKcq4uvzh54TpSOHqhJixJMF8HoBuFgT9yIwWZDpE9nCt6Ygh0v\n6ukuKM6COgsZf/99jKQeBmV2mblxy8dH1u4/TioCTyfVKHk87QWmm9gLEyBU\nS8iMEpoEyza1Zb7+PvknZEbsUWjmuo0+yKQqvsTIp5n8H23JmfHIsPHy7rjw\nimQgN2m9Izh4iFucL90ciUQ321gz1+losczgVKleej/ZjJ3A2WG+cCW6X0MV\nXK+0Ddv9SyqngHmXccjyWSRcBknzNNdnNgmxFG+fiHC7haoPgDnNKWykDAg5\nNGfD8wwJe1QDScvdky/hMtSeQopGiqm3Jcxbnlc9QLtdm5XNoDXES6dAaGCI\nmX2BPwcrRXrWPMOES+gUh0SJRLxQCY7OQniUzb9+SHcuutmdH4WiYzSCtr5X\nHLkCGtda8icDajbZVwwicfqgDKNDyEdmrECnHmsBqwEYjcE1+HLCYf24RuXB\n/ZD8\n=TyGB\n-----END PGP SIGNATURE-----\n"},"architecture":"AMD64"}]

dilshat commented 6 years ago

@emli @marcoarthur guys lets check the code used for version comparison

emli commented 6 years ago

1) The first bug, info rest doesn't return lastest version, it returns latest by date. 2) The second bug, a user can upload the same version of template twice.

dilshat commented 6 years ago
  1. The second bug, a user can upload the same version of template twice.

See issue description for the exact algorithm

lbthomsen commented 6 years ago

@emil - I definitely uploaded ubuntu-xenial 0.1.10 after 0.1.9 so that doesn't seem right.

lbthomsen commented 6 years ago

Still allow multiple uploads of same version which isn't exactly a problem, but if multiple verssions exist it should pick the latest - that does not appear to happen.

samsonbek commented 6 years ago

IMO, deleting(replacing) file implicitely when same version uploaded is wrong approach. File owner has to explicitly delete it. Or, we need some function called "Replace" for such actions, instead of "Upload". To download correct file, we may introduce params like latest version or last uploaded.

emli commented 6 years ago

If two templates with same version with same user, info rest will return by latest by date.

dilshat commented 6 years ago

@emli for the same user and same template name there should be single version. Lets fix this @samsonbek I never saw two files of the same product of the same version with different hash sums in any repo.

dilshat commented 6 years ago

For example we have automation script that uploads the same artifact frequently to test it. If we dont replace but append, older revisions of the same version would occupy the disk.

We have to either then replace or add revision of version concept. It will be visible only to Kurjun.

dilshat commented 6 years ago

If we choose replacement option, then we have to make sure that when artifact is being uploaded, it is saved into one place, and only after upload is complete, the original one is replaced. To avoid broken partial artifacts

emli commented 6 years ago

How to test:

1) Upload template from hub 2) https://eu0.mastercdn.subut.ai:8338/kurjun/rest/template/list?owner={Put_your_finger} - to see your templates, check your template here 3) Upload template from hub same template 4) https://eu0.mastercdn.subut.ai:8338/kurjun/rest/template/list?owner={Put_your_finger} - to see your templates, check your template here, you shouldn't see old template that you in step 2.

AlinaPenkina commented 6 years ago

Fixed on master.

dilshat commented 6 years ago

Happend again