List and delete deployments

[Nowaker]

I've just noticed by Firebase hosting usage is now almost 1 GB. Quite a lot given the fact our website is only 20 MB.

nowaker@nwkr-desktop ~/projekty/virtkick/website (git)-[master] % du -hs build 
20M     build

It seems all previous deployments are kept by Firebase and they're visible at https://console.firebase.google.com/project/PROJECTNAME/hosting/main.

Deleting 90 deployments one by one manually is out of question. There's a big need to have a way to list deployments via CLI and delete them.

[brendanlim]

@Nowaker This is definitely on our radar and something we're looking to improve

[zhenhaolei]

hey @brendanlim any updates here? We are also seeing infinite loading in the hosting module, looks like we have too many deployments :(

Thanks!

Error: too_big: The data requested exceeds the maximum size that can be accessed with a single request. at r (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8597) at H (third_party/javascript/firebase/firebase_js_minified.jslib:126) at Object.eval [as H] (third_party/javascript/firebase/firebase_js_minified.jslib:206) at eval (third_party/javascript/firebase/firebase_js_minified.jslib:190) at Kh.g.Id (third_party/javascript/firebase/firebase_js_minified.jslib:196) at yh.Id (third_party/javascript/firebase/firebase_js_minified.jslib:186) at qh.eval [as zg] (third_party/javascript/firebase/firebase_js_minified.jslib:184) at th (third_party/javascript/firebase/firebase_js_minified.jslib:178) at WebSocket.ua.onmessage (third_party/javascript/firebase/firebase_js_minified.jslib:177) at WebSocket.b [as __zone_symbol___onmessage] (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8568) at w.invokeTask (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8611) at u.runTask (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8601) at WebSocket.invoke (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8613)

[Nowaker]

@brendanlim @mbleigh Any update on this? Our deployment history is constantly growing, each CI build at a time, and it's beyond our human capabilities to keep up with deleting them!

[mbleigh]

We're aware of the issue here and are considering the best way to tackle it. In general we don't want you to feel worried about having a growing history of versions. Curious for anyone who comes across this (vote with emoji on this comment): which would you like best?

• :tada: The ability to list and possibly batch delete old versions
• :+1: Old versions are only kept for a certain amount of time unless "pinned" in some way
• :heart: Only a certain number of old versions are kept unless "pinned" in some way

[Nowaker]

"The ability to list and possibly batch delete old versions" is a primitive. APIs are built using primitives. These primitives allow users to achieve whatever they want without bugging anyone (in this case, you @mbleigh).

The other two are high level features. They're cool. But then comes a question: how do I pin or unpin the version via API or firebase-tools? As you can imagine, once again someone will have to click through the list in Firebase UI one by one, just like to delete old deployments today. :rofl:

To sum up, high level features are great and needed, but CRUD primitives in API are super important for tools like Google Cloud Platform or Firebase.

[mbleigh]

@Nowaker I don't disagree, and both are important. It's fair to say it's a false dichotomy between the first option and the other two. FWIW, in order to implement the second two options we'd pretty much have to implement everything necessary for the first anyway.

This stuff is definitely on our radar, but I don't have specifics on when we'll have something to show.

[CaptainCodeman]

I would make do with a checkbox to select all / multiple entries on a page and delete them in one go. Also, make it possible to dismiss the confirmation dialog by pressing enter to confirm.

[WhatsThatItsPat]

You could also not hide the delete functionality. Right now you:

1. Rollover deployment row.
2. Click the three dots menu.
3. Click delete to call up the confirmation.
4. Click delete again.

If each row had it's own delete button, and if you could hold shift to bypass the confirmation, you could fly though them.

[titpetric]

Is an REST API for listing and deleting deployments available, if one wanted to implement this functionality on their own (or even implement it in firebase-tools, and submit a PR)? If the endpoints are just not there, I can see how that might be a blocking issue. Is this on any kind of road map with something similar to a due date?

[Thkasis]

Any news on this? It would definitely help to have a simple way of removing previous deployments in CLI. Thanks for the good work.

[mbleigh]

No news on this yet, but it's an area of active interest for the team. Thanks for your patience!

[shaun-sweet]

Any news on this? We have so many we have to manually remove, one by one.

[glennlawyer]

Would also really appreciate some progress on this!

[mbleigh]

We're doing a lot of work on our deployment infrastructure right now that will take a bit of time to come to fruition. This issue is definitely on our mind as we do that work.

[titpetric]

@mbleigh hey how about locking this topic. I'd like to be notified when it's done, but I really don't want to read all the "me too" comments.

[ludvig-larsson]

"Only a certain number of old versions are kept unless "pinned" in some way" is exactly what we need.

[rtm]

The other problem I have is that I when I look at this list of deployments, I have no idea which is which. I version my app in a couple of ways, including the version in package.json, and an notion of "builds", so I could also provide a version and/or build # to firebase deploy, and then if that could be listed in the list of deployed versions it would be fantastic. As it stands, I see 100 deployed versions and have no idea which is which.

[a-xin]

@rtm You can specify a message for deploys which is shown in the list of deployments:

firebase deploy --message "build 1234"

The option is not listed on the documentation pages, but it's listed when running: firebase help deploy

[rtm]

@a-xin Thank you. I had totally missed that and it's very useful!

[haraldreingruber]

It would be great if the priority of this issue could be increased, as we do continuous deployment of every git commit, and each deployment is a couple of hundred MB. Would be nice if we were able to integrate some commands for cleaning up old deployments in the CD script as well, in order to reduce the overall storage consumption.

[dinvlad]

As a workaround, would it be possible to reuse the files whose filename and contents haven't changed (according to some hash)?

E.g. if Webpack generates chunks with stable hashes (e.g. employing HashedModuleIdsPlugin), do these files get uploaded on every deploy (even though they truly haven't changed)?

That could significantly cut down on the amount of file updates on each deployment (down to kbytes of our application code, if vendor libs haven't changed).

[mbleigh]

This is not a low priority for us. We are working on the enabling infrastructure to tackle this issue now, but there are large changes necessary to do what we want to do. It will take some time, thanks for the feedback and your patience.

On Sun, Feb 25, 2018, 3:32 PM Denis Loginov notifications@github.com wrote:

As a temporary workaround, would it be possible to reuse the files whose filename and contents hasn't changed (according to some hash)?

E.g. if Webpack generates chunks with stable hashes (e.g employing HashedModuleIdsPlugin), do these files get uploaded on every deploy (even though they truly haven't changed)?

That could significantly cut down on the amount of file updates on each deployment (down to kbytes of our application code, if vendor libs haven't chagned).

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/firebase/firebase-tools/issues/215#issuecomment-368355645, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAD_nbNy4j3gsIpTk_YpLyhlAuPtNHOks5tYe2DgaJpZM4J0BKU .

[sejr]

I am glad to see that this is a priority. I am on the Spark plan and have a web application that is somewhere around 20MB. I have been a little liberal with my deployments so I have 300 to go through and delete. 😅

In addition to the limited # of deployments in history, I also like @dinvlad's idea of reusing files.

[sboyd]

@sejr one thing I do that helps is use the keyboard as much as possible. I click the elipses, hit the down arrow then enter, then the tab button 3 times then enter And repeat.....hopefully that helps.

[billiaug]

@mbleigh Any news or progression on this?

[r1th4l1n]

@mbleigh can you give us a status?

[haraldreingruber]

@mbleigh Looking forward to more transparency. Maybe you could elaborate more about the general firebase-tools roadmap, and the plans regarding this issue? (It's the highest voted issue btw, almost double the votes compared the second highest one).

[mbleigh]

Sorry for the long delay on addressing this -- it's high on our list, but we've been heavily investing in an infrastructure project that sets the foundation to take on many smaller projects like this one.

I can't promise exact timelines, but this definitely hasn't been forgotten.

[Nowaker]

Maybe not forgotten but surely ignored. A deployment can be deleted in the UI with a couple clicks - there's no reason why the same couldn't be easily exposed via API. I'm extremely disappointed by Firebase product roadmap, as well as the fate of Divshot.

[mbleigh]

I understand the frustration, truly. This issue is being addressed by work we're doing now and hope to bring to all of you in the near future.

On Fri, Jun 29, 2018, 1:01 AM Damian Nowak notifications@github.com wrote:

Maybe not forgotten but surely ignored. A deployment can be deleted in the UI with a couple clicks - there's no reason why the same couldn't be easily exposed via API. I'm extremely disappointed by Firebase product roadmap, as well as the fate of Divshot.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/firebase/firebase-tools/issues/215#issuecomment-401279887, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAD_t_BTD_cT8eWOABg8sfs9Lf1n9g8ks5uBd7SgaJpZM4J0BKU .

[rtm]

@mbleigh

but there are large changes necessary to do what we want to do.

Maybe the seething masses would be pacified by some suitably vague hand-waving about what these large change are, and what other earth-shattering features they would enable. Failing that, it's not unreasonable for some folks to get their undies in a twist about what does seem like merely ignoring the feature, since it's going on almost two years now.

[mbleigh]

It's pretty straightforward -- Firebase Hosting has never had a public API, and the current method for listing and deleting deployments in the console is not well-suited to publicizing. Since the CLI is open-source, we need to make sure any actions exposed through it are appropriately scalable.

We're in the process of migrating a large portion of Firebase Hosting onto Google infrastructure that will make it more scalable and also provide a strong foundation for properly having APIs for things. This effort is very good for the long-term health of Firebase Hosting but has been a substantial investment for the team that has resulted in a long period of no apparent progress.

I appreciate that many of you feel strongly about this -- if you reach out to support, we're happy to manually prune your older deploys in the interim. We hope the wait won't be much longer, but as with any software, picking exact dates for things is usually a bad idea.

I hope this clears things up a bit, and thanks to all Firebase Hosting users for bearing with us 😄

[haraldreingruber]

@mbleigh Thanks a lot for the heads up.

[ghost]

@mbleigh I'm not sure why you need to make this task dependent on a public API or any infrastructure change.

This can be solved in the short term with a button in the Firebase console "Prune old deployments".

Since the console is already making calls to the backend, it can get a list of all deployments and then call delete. I'm assuming the API on the backend doesn't take a batch list of deployments, so this button might cause some issues if you do it synchronously with one deployment at a time.

Anyways, the point is that don't over engineer a solution when all you need to do is add a button to an already established API in your backend. Especially, don't make your users wait 2 years for it.

Not professional at all.

[HenrikBechmann]

I don't know what everyone is grousing about. It's RELAXING to delete old deployments one-by-one. I just cleared a backlog of 72. Then there was double checking to make sure I'd gotten them all (I missed one) -- again very satisfying. I felt virtuous, and dare I say SUPERIOR. Taking care of details; due diligence. Administrative fastidiousness! My only regret was that I didn't have MORE to clear up. I actually had to play solitaire for a WHOLE HOUR afterwards, just to feel well enough to move to my next task.

And two years (and counting) to get this feature?!? Pfftt. Everyone knows that engineers have to take CARE to do these things right! And it's a huge BUDGET that's required. I mean think of all the effort WAYMO and the rest of Alphabet is taking. Google couldn't POSSIBLY afford to put more resources into this feature.

And besides, we're just DEVELOPERS! DOUBLE Pfftt. How do you expect Google to survive by putting DEVELOPERS at the head of the queue?!? I mean really!

No, for the good of us all, and particularly for our well-being, I personally vote that the dev team here continue to work on this for at LEAST another two years before implementing this feature. For God's sake be careful! Don't be rash; don't be rushed!

And for heaven's sake, don't do anything impulsive like having a temp feature of a quick delete button or some such; that would just divert resources from much more important things.

All good as is; please don't make ANY changes to your process, such as, say MANAGEMENT changes!

Oops, gotta go. Time for my nap (I'm exhausted from all the concentration required to delete all those deployments). And besides with such a productive time, I certainly deserve some time off!

[sanjeevhalyal]

I have 125 of them to delete.

[mbleigh]

🤔 Somehow I feel like there's a bit of sarcasm happening in this thread... 🙃

Folks, I really do get the frustration. To reiterate a statement from my previous comment: if you reach out to support, we're happy to manually prune your older deploys on your behalf in the interim. Just let us know how many old versions you would like to keep around.

As for why we're not releasing an interim band-aid solution -- well, the support intervention is our band-aid. We'd prefer not to burn limited engineering resources on a system we're actively working to replace, which is why this ends up in a bit of a tough spot.

[mbleigh]

Oh look, someone seems to have dropped a gist that might be relevant to this thread...

P.S. We're still working on a better, automated solution but in the meantime this is a much better band-aid. 😼

[haraldreingruber]

Thanks for taking the time to provide a work-around.

[anatesan-stream]

The script in the gist above works fine in terms of deleting the versions, but I don't see my Storage Usage come down (does it take a while to recompute this after the deletes of the versions?)

[zevdg]

does it take a while to recompute this after the deletes of the versions?

In my experience, yes. Even if you use the gui to manually delete a bunch of versions, it takes some time for the usage numbers to change.

[wahidyankf]

Really thanks @mbleigh for providing us with a much better band-aid. :pray:

[alexanderwhatley]

Any updates @mbleigh?

[jackcw]

Have you looked at the hosting rest api @alexanderwhatley its not a ui solution but the new Hosting Rest API should enable you to do everything you need to trim up your deployments

Hosting REST API

[ghost]

Have you looked into it @jackcw ? Cause I can only find create and list methods. No delete method.

[jackcw]

I havent actually used it but as far as i can see there is a delete on the versions endpoint and the list on releases includes the version object so i assume you do a list of releases and take the version id and hit the version delete endpoints

[ghost]

Sorry my bad. I misunderstood the meaning of versions and releases. You can now delete older releases which are called versions with the API.

[andipaetzold]

I created a small shell script that is executed once a day with a cron job to remove all old releases. Here is the gist. It requires jq to parse the JSON. It basically does what @jackcw wrote: It iterates over all releases and deletes them.

I am not an expert in writing shell scripts or jq - but the result is what counts, I guess. The script works quite reliably for me. Feel free to use it yourself.

[mbleigh]

Internal tracking id: 113235359

[paulmand3l]

Any updates on this?