search

recoilme / pudge

Fast and simple key/value store written using Go's standard library
MIT License
371 stars 35 forks source link

Overwriting old values does not work! #16

Closed michail-vestnik closed 1 year ago

michail-vestnik commented 1 year ago
На русском

Пожалуйста помогите решить проблему. Я прочитал что ваш интерес к проекту пропал и всё внимание уделяется sniper. Но мы встроили pudge в проект, где он идеально вписывается и поздняк метаться. Но pudge не перезаписывает значения. Прилагаю скриншот. Слева код а справа база данных.

On english

Please help solve the problem. I read that your interest in the project has disappeared and all attention is paid to sniper. But we built pudge into the project where it fits perfectly. But pudge doesn't overwrite the values. Here's a screenshot. On the left is the code and on the right is the database.

Снимок экрана от 2023-04-05 09-28-12

recoilme commented 1 year ago

it's not a problem in general, most of databases don't remove old values physically from file (Sniper does this)

recoilme commented 1 year ago

https://github.com/recoilme/pudge/blob/master/api.go#L514 backup must clean from unused space, for example

michail-vestnik commented 1 year ago

Thank you for your time! I like sniper, it's similar to https://github.com/VictoriaMetrics/fastcache. But I took the pudge engine, heavily modified it, added cuckoo filter, etc.. So it's a shame to lose the work I've done.

recoilme commented 1 year ago

Why it's critical? Many overrides, huge db size? Maybe just a repack db from time to time as we do with postgres and other db's?

michail-vestnik commented 1 year ago

I think this is critical for metrics and statistics. For example, the number of views and visits. Do I understand correctly that deleting is a very expensive operation because you have to overwrite the file every time? Is the byte overwrite operation expensive?

recoilme commented 1 year ago

For example you override key sophia with ekaterina. You have file: [...Sophia... 10Mb of data] - and try to replace Sophia with Ekaterina, but no room for Ekaterina and you must rewrite all 10Mb

In case of metrics we must use 8byte room for any int and override just in place (find address/get value/increment/override) - but it need some specific method for this case, something like IncrInt. You may fork and implement it

michail-vestnik commented 1 year ago

Thanks for your time! you helped me