last_activity_at is restricted to one update per hour #21

Open NicoCa opened 7 years ago

With GitLab CE Merge Request !6391 the last_activity_at column is restricted to one update per hour.

Since the packages.php script relies on the last_activity_at value to determine if the packages.json needs to be updated in general and which project in perticular, you may now get an "outdated" packages.json if you push frequently.

Hello,

Thanks for letting us know. However, I fail to see any way to bypass this problem, short of removing the caching system entirely, which doesn't seem to be an appropriate option.

Do you have any ideas on how to address the issue? In my opinion, short of having a notice in the documentation to warn about the short-coming, I don't see many solutions.

Any suggestion, or even better, PR, are always welcome!

You could grab created_at from commits. It looks like currently only branches and tags are searched. If that was intentional, they both have a committed_date which could possibly be used.

If I have time later I will test and submit a PR. =)

That's an interesting idea. Yes, only branches and tags will ever be searched since it's pretty much the only way to provide a release (either stable through tags or dev through branches).

Hi there guys,

any news about this?

Cheers

It looks like @SautdeChat didn't follow up on their future PR. I would encourage you to have a look and send one if this is a feature you really need ;).

The solution for our system ended up being a less impressive "Disable the cache." Our repositories are much more frequently updated than requested, so this solution works for us. Sorry I did not update. I can confirm that the updates are still once per hour. One could loop through all the projects, get the commits, and use their creation dates to build the cache. But this will greatly increase generation time due to many more API calls. Perhaps longer than generating it fresh each time. A better solution would be to add a System Hook (API), which would make a POST request from GitLab upon certain events. The System Hook can be created via the API, so this should not increase complexity for the end user. As a side note, this project uses APIv3 by default, which is deprecated as of 9.0, and will be removed in 9.4.

Happy to help out with either issue, once a preferred solution is decided.

Thank you for the follow up!

First: "This project uses APIv3." Unfortunately, has you might have noticed, this project is not really actively maintained. I will take a look at any issue or PR opened, when I get time. Unfortunately, I do not have the time to maintain it myself. I'm sorry about that. Any PR aiming to upgrade the API version is more than welcome.

PSA: If you ever feel like becoming maintainer, then, PLEASE, reach out privately. I would be more than happy to help provide a second life to this project. Any project in the wemakecustom organisation with an open-source licence is ready and eager for the transition ;). In the mean time, it will unfortunately remain community supported.

On the issue at hands: The System Hook sounds like a really nice idea! It could allow a much better cache control! I'd say that's definitely the way to go. Again, if you have the time and motivation, I'm really open to PR. Sorry I can't really do more...

Guys, at the end, the solution we found was use satis and some php based on this one grabbing all the repositories for generating the config.json file provided.

Cheers

This issue can be solved by solving https://github.com/wemakecustom/gitlab-composer/issues/32 for what recently a pull request was created: https://github.com/wemakecustom/gitlab-composer/pull/36

I also had a look at satis and gitlab to satis configuration export and satis is also using a cache(the composer cache) and running it periodically every minute with clearing the cache must also have the same performance issues, don't it?

wemakecustom / gitlab-composer

last_activity_at is restricted to one update per hour #21