Allow expired short URLs to be programmatically deleted

[ppfeiler]

Summary

It is currently possible to set a "validUntil" property when creating short links. Once the "validUntil" date is reached, the short link ceases to function, though it remains in the database.

It would be beneficial to automatically delete all expired short links after a specified timeframe.

I acknowledge that there may be use cases where individuals prefer to retain expired short links, so this functionality should be optional and configurable during the creation of such links.

Use Case

We generate numerous short links via the API, each with a validity period of 10 days. Once this period elapses, the links become invalid, rendering them obsolete.

While this is currently manageable, it presents a potential issue as the database size will rapidly increase. We are generating approximately 25k short links per week, all with a 10-day validity period.

[acelaya]

These are some of the other requests for this feature:

• https://github.com/shlinkio/shlink/discussions/1710
• https://github.com/shlinkio/shlink/discussions/1964

Even though I agree this would be definitely a useful feature, it is very tricky to implement, considering the architectural requirements it would have.

For example, Shlink supports (and always will) the classic approach to run and serve PHP apps, using a traditional web server and fast-cgi process manager. In that context, it's very hard to implicitly schedule tasks.

The other supported approach, which is RoadRunner, does have a concept of background jobs that can be programmatically scheduled to be immediately executed, but not at a particular time.

Basically, the logic that takes care of deleting a short URL that has expired, would need to be scheduled somehow, making sure it's deleted only if expired, and take into consideration that the short URL could be edited at any point in time, removing, delaying or forwarding that date.

One could also argue if short URLs which reached maxVisits should also fall into the "expired short URL" bucket.

Since Shlink does not currently have a way to run code without it being triggered by an incoming request or console command, I can propose an alternative approach. I could add a new console command to Shlink, that would allow you to delete all existing short URLs that can no longer be visited, whether it is because of validUntil date in the past or maxVisits has been reached. Then, it's up to you to schedule that command to be run periodically (let's say once per day) using the task scheduler of your choice (let's say cronjobs).

That would be reasonably easy to implement in a backwards-compatible way, and remove all the tricky parts and edge cases away from Shlink.

The command could look like shlink short-url:delete-expired [-f] [--evaluate-max-visits].

[ppfeiler]

Thank you for your response. You're right, given the current architectural constraints, a console command is the best solution.

You're also correct about the short URLs that have reached "maxVisits." Those should be deleted as well.

I'd like to propose one more suggestion for discussion: there might be short URLs that shouldn't be deleted even if they have expired. For instance, if they're used for a marketing campaign that gets reactivated periodically.

My suggestion is to introduce a property called "mark-for-deletion" (I couldn't think of a better term at the moment) when creating a short URL. Then, the console command would only consider those that have expired and have "mark-for-deletion" set.

What do you think of this proposal? Do you have a better idea for marking expired short URLs so they won't be deleted?

[acelaya]

I think that would be problematic and quickly become confusing for users.

• If you are marking the short URL as non-deletable, people will think deletions happen automatically. A property that means "it is not auto-deletable, but only applies if you do something else, otherwise this does nothing", will be very hard to reason about.

Instead my thinking is as follows:

• If you don't mind about old expired short URLs, you just ignore the console command, and everything keeps working as it has so far.
• If you use Shlink to create many temporary URLs and want to periodically delete them, then you use the new command. Shlink assumes all "expired" short URLs are safe to delete.
• If there's any exception to the point above and you want to temporarily hide a short URL but without considering it "expired", it is more intuitive to set it a validSince date in the future. That would keep it safe from the new command.

[acelaya]

As of https://github.com/shlinkio/shlink/pull/2086, this is now implemented

[acelaya]

Shlink 4.1.0 has just been released, including the new short-url:delete-expired console command.