Refactor Package model to have descriptor and readme

[zelima]

Currently we are getting README and datapackage.json for package from PackageTag table. We have OneToMany relationship between Package and PackageTag tables. We need them to get it directly from Package table

Acceptance Criteria

• [x] Able to retrieve metadata for package with simple query on only Package table
• [x] New tests for models

Tasks

Task list for a non-epic issue

• [x] Write new tests [1]
• [x] Refactor models and update old test where appropriate[1]
• [x] refactor publishing part in logic [1]
• [x] refactor getting package part in logic [1]
• [x] refactor tests where appropriate [1]

Analysis

class Package(db.Model):
    __tablename__ = "package"

    ...
    tags = relationship("PackageTag", back_populates="package")

class PackageTag(db.Model):
    __tablename__ = 'package_tag'

   ...    
    descriptor = db.Column(db.JSON)
    readme = db.Column(db.TEXT)
    package = relationship("Package", back_populates="tags")

We want to have descriptor and readme columns in Package table as well, that will by default be same as latest updated tag in PackageTag table.

• Unless tag is explicitly set we are not touching PackageTag at all
• dpm publish populates Package table only (with appropriate dp json and README) (tags column remains empty)
• dpm tag populates PackageTag table only (tags column in Package table gets populated)
• only dpm publish [tag] publishes data under the given tag

[rufuspollock]

@zelima looks pretty accurate:

• I don't think we want a PackageTag for current (latest). Tags are versions that are explicitly created (see the user stories). => Whenever we tag (with whatever tag name) we create/update the PackageTag with current state and copy in BitStore too. (note: strictly we could even leave most info e.g. readme, descriptor off PackageTag since we have it in the BitStore - have in the DB is just caching really).
• It would be good if you linked from your tasks to the relevant parts of the code e.g. "refactor publishing part in logic" could be link to part of logic you are refactoring.

[zelima]

@rufuspollock this confused me a little bit.

• So unless tag is explicitly set we are not touching PackageTag at all? all the inf goes into Package table correct?
• Are not we actually tagging with latest behind the scenes? should we get rid of this action?
• I'm loosing the track of why do we need PackageTag table at all

[rufuspollock]

@zelima

So unless tag is explicitly set we are not touching PackageTag at all? all the inf goes into Package table correct?

Yes

Are not we actually tagging with latest behind the scenes? should we get rid of this action?

Yes, I'd get rid of this. "latest" is not a tag really. Think of git(hub) - "master" HEAD is not a tag - tags are things you have explicitly created.

I'm loosing the track of why do we need PackageTag table at all

How do you create tags? Read the user story again re tags: they are like git tags - a way to label (and make a copy) of the data package at a given point. We need somewhere to store that info.

BTW: it sounds like adding documentation about Tags/Versions to the main documentation explaining these ideas and the implementation could be useful.

[zelima]

FIXED. tags are explicitly created. Metadata is saved into Package table