search

excieve / dragnet

Catching the big fish
MIT License
2 stars 1 forks source link

Temporary variables from CoucbDB views are leaking into DB #6

Closed dchaplinsky closed 6 years ago

dchaplinsky commented 6 years ago

Or so it seems.

I've compared results of old and new export on the same set of documents, here is the diff: https://gist.github.com/dchaplinsky/1c84a0f47d8ea65b5808939ba15942a3

dchaplinsky commented 6 years ago

rev and other couchdb fields are expected, difference in data format of aggregated fields is expected too.

But pay an attention to records like that: https://gist.github.com/dchaplinsky/1c84a0f47d8ea65b5808939ba15942a3#file-decls_100000-diff-L115

excieve commented 6 years ago

These are not temporary and not leaking. In fact, they're not coming from views but actually from the import preprocessing. So that's expected.

dchaplinsky commented 6 years ago

Is there a way to at least prefix them?

excieve commented 6 years ago

That's certainly possible. Will work something out if it's causing problems.

On 4 November 2017 at 13:23, Dmitry Chaplinsky notifications@github.com wrote:

Is there a way to at least prefix them?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/excieve/dragnet/issues/6#issuecomment-341889419, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBvyb0Y0JbYiExM51tXpXNO9wQDqg9Hks5szEktgaJpZM4QRExu .

dchaplinsky commented 6 years ago

Well. This data is later provided through API and data dumps and it's a bit unpleasant to supply such an altered data to end users without at least warning them. Prefix might help here.

dchaplinsky commented 6 years ago

Thanks