search

backdrop-contrib / bee

:bee: Bee is a command line utility for Backdrop CMS. It includes commands that allow developers to interact with Backdrop sites.
GNU General Public License v2.0
16 stars 20 forks source link

Add 'update' command #111

Open ghost opened 3 years ago

ghost commented 3 years ago

I just realised that we don't have an update command for updating a module/theme/layout to a new version. We should add one.

Work is in progress on the issue-111-update-command branch. Huge thanks to @TheMetMan for getting this feature started.

The current branch should work ok for a single site, or a multisite that only uses site specific module/theme/layout folders. It will download all updates.

⚠️ WARNING: It is not recommended to use this branch directly on production until #297 is merged. ⚠️

cptX commented 3 years ago

I just installed bee, and the first thing I noticed is this functionality missing. We definitely need this!

laryn commented 2 years ago

+1

It would be nice to allow to specify one or more items to update, or just to use a global update command to update all that have new releases. I think on Drupal you could do (for example) drush up and it would check and update all; or drush up paragraphs or drush up drupal to pinpoint the update you want to do. Also potentially useful: a flag to say only do security updates.

yorkshire-pudding commented 2 years ago

+1

As a starter for ten here is draft help based on @laryn 's comment above. This could form requirements:

update Description: Updates backdrop, modules, themes and layouts with new releases Aliases: up , pm-update Arguments:

If I wanted to have a go at doing this (no promises as not sure that I can), is there a good place to start?

ghost commented 2 years ago

@yorkshire-pudding The new command would go into https://github.com/backdrop-contrib/bee/blob/1.x-1.x/commands/update.bee.inc

See https://github.com/backdrop-contrib/bee/blob/1.x-1.x/API.md for an explanation of the various options when creating commands.

yorkshire-pudding commented 2 years ago

Thanks @BWPanda - that is really useful. Is this the right bit of core api ? https://docs.backdropcms.org/api/backdrop/core%21modules%21system%21system.updater.inc/1

yorkshire-pudding commented 2 years ago

~~if anyone is interested or would like to help, I've started work here: branch removed So far, all I've done is add the details to update_bee_command and create a skeleton update_bee_callback function with a long list of todos. Even if its just to let me know if I'm on the right lines:~~

/*  
* @todo consider if any drush extension code can be re-used https://github.com/backdrop-contrib/backdrop-drush-extension/blob/1.x-1.x/commands/pm/backdrop_pm.drush.inc
  * @todo consider if brush can be re-used https://github.com/backdrop-contrib/brush/blob/1.x-1.x/commands/pm/updatecode.pm.inc
  * @todo check if file permissions allow backdrop to update itself? 
  * @todo determine required files:
  *   - core/modules/update/update.fetch.inc
  *   - core/modules/system/system.updater.inc ?? (https://docs.backdropcms.org/api/backdrop/core%21modules%21system%21system.updater.inc/1)
  *   - core/includes/update.inc
  *   - core/modules/installer/installer.manager.inc (https://docs.backdropcms.org/api/backdrop/core%21modules%21installer%21installer.manager.inc/1)
  *   - core/modules/installer/installer.manager.inc (https://docs.backdropcms.org/api/backdrop/core%21modules%21installer%21installer.manager.inc/function/installer_manager_update_ready_form/1)
  * @todo process arguments (list of projects)
  * @todo process option (security-updates-only)
  * @todo check for config item - update_disabled_extensions (Configuration>Update settings>) - if this is true (1) then need to check disabled projects also
  * @todo get list of installed projects (core, modules, themes, layouts) if no argument list of projects
  * @todo list of projects is either all installed + core, all enabled + core or argument list of projects
  * @todo get current backdrop version
  * @todo for each project in list get current version, type, name
  * @todo for each project in list get available version
  * @todo compare versions and prepare list of possible updates
  * @todo ??option to review and proceed or cancel??
  * @todo download updates to temp directory
  * @todo install updates
  * @todo can we / should we check if database updates needed and let user know
*/
TheMetMan commented 2 years ago

This is a really useful addition in my humble opinion. Drush does not work in PHP 8, so I need to move on. I would like to help if I can. I use PhpStorm for my own code development. I do not seem able to get the debugging to work and step through Brush or Bee. For Bee I have the arguments as follows --root=/home/francis/FG-Docs/public_html/moduledevbd.scorpio/web status But when I run it it does not pick up the Arguments as shown by this error:

Invalid argument supplied for foreach()
 /home/francis/FG-Docs/PhpstormProjects/Bee/includes/command.inc:13

The $argv variable is null Would anyone be willing help me get it working? Or am I asking in the wrong place being new to helping on GitHub

yorkshire-pudding commented 2 years ago

@TheMetMan - if you navigate to moduledevbd.scorpio and then do bee --root=/web status does that work?

I'm afraid I kind of develop in disjointed way where I edit (in visual studio code) then test in lando so not sure how to setup for debugging. My initial guess is its struggling to parse a long path (or something in the path) but don't quite have time to investigate at the moment. Might do later this evening.

TheMetMan commented 2 years ago

Thanks for the quick response. I am a Linux man by the way. I always find it useful to step through the code. Perhaps consider porting the update from Brush to Bee

TheMetMan commented 2 years ago

I should have said I am trying to run the debug in the PhpStorm IDE, not from the command line, I will see if Jetbrains can help

yorkshire-pudding commented 2 years ago

@TheMetMan I remember your presentation at Backdrop Live so I have a good idea of your setup. My thoughts in my initial statement was maybe about checking to see if bee could cope with the paths. Perhaps try that long path in the command line so you can see whether the issue is in bee or in the IDE.

TheMetMan commented 2 years ago

I remember you too. Judging from your accent and the 'handle' you are from Sunny Yorkshire. I was born in Sheffield, but live on the very edge of South West Ireland with a nice view into the Atlantic. I am beginning to think I have to setup the SERVER variable in the Environment. I am using a Virtual Host, so I can add a print statement to see what I get when I run it from the command line first.

yorkshire-pudding commented 2 years ago

You might get more help on integrating with PHP Storm in Zulip as you'll reach a wider band of people who probably use it.

TheMetMan commented 2 years ago

Very good idea, thanks for your time. Regards

TheMetMan commented 1 year ago

@yorkshire-pudding, much to my surprise, I have got a very basic update working to update the modules and themes, not backdrop itself (already taken care of in the commands). I have created it on my account here https://github.com/TheMetMan/bee if you fancy a look. running this comand: bee update will update any modules and themes which need updating, no questions asked.

TheMetMan commented 1 year ago

I have tried to add a prompt Y/n before doing the actual updates so you can abort, but I cannot get it to work. It is there, but the script does not display the prompt. The script appears to hang.

yorkshire-pudding commented 1 year ago

Thanks for this @TheMetMan . I have had a brief look, and will have a more detailed look and do some testing later.

I have tried to add a prompt Y/n before doing the actual updates so you can abort, but I cannot get it to work. It is there, but the script does not display the prompt. The script appears to hang.

Is your fork of bee up to date with the main repo? What version of backdrop are you on? There was an issue a little while back affecting prompts so would be good to rule out that this isn't the cause here.

TheMetMan commented 1 year ago

I am using #267 version of bee and I am using 1.22.0 on Backdrop

yorkshire-pudding commented 1 year ago

If you're using PHP 8, then you will need 1.24+ or 1.23.2 for the confirmation prompts to work on bee - there was a temporary workaround while we got core fixed, but that workaround has now been removed.

This may not be the case here but worth ruling in or out.

TheMetMan commented 1 year ago

OK @yorkshire-pudding , I will check it out.

TheMetMan commented 1 year ago

OK, @yorkshire-pudding, the prompt works in 1.24.0, but the messages prior to the prompt have not yet displayed. I cannot see how to force the messages to display prior to the prompt. This is because I am a bit of a beginner at this.

yorkshire-pudding commented 1 year ago

@TheMetMan - I'll have a look later

yorkshire-pudding commented 1 year ago

@TheMetMan - sorry this is taking a while; I've tested a couple of scenarios, some more to check before feeding back, but day job is getting in the way.

TheMetMan commented 1 year ago

Ok, Martin, I am away on hols, so no rush at all. Cheers

On March 13, 2023 at 11:28 AM Martin Price @.***> wrote:

@TheMetMan https://github.com/TheMetMan - sorry this is taking a while; I've tested a couple of scenarios, some more to check before feeding back, but day job is getting in the way.

— Reply to this email directly, view it on GitHub https://github.com/backdrop-contrib/bee/issues/111#issuecomment-1465972224, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOLWRJZ22SDY3YZLO6UBN3W34AFVANCNFSM4UCEY3NQ. You are receiving this because you were mentioned.Message ID: @.***>

marioawad commented 1 year ago

I have got a very basic update working to update the modules and themes, not backdrop itself (already taken care of in the commands).

@TheMetMan Can you please point me to the command that you use to update backdrop itself?

I tried using download-core but it refuses to download backdrop in the dir of the website. So currently to update I'm using download-core to an external dir and then manually moving the core dir to the site I want and then finally using update-db which is working really great.

Thank you for creating bee!

yorkshire-pudding commented 1 year ago

@marioawad - this is not ready yet as a bee command. I need to worth with @TheMetMan and hopefully we will have this ready for release at some point.

bee has been created by a number of people over the years. See https://github.com/backdrop-contrib/bee/graphs/contributors for a list of all contributors who have had code merged.

marioawad commented 1 year ago

Thanks for the prompt reply. Do you know of any alternative to update backdrop core from the command line please? Also, where do you recommend I look around to try and help you out if I can?

yorkshire-pudding commented 1 year ago

Do you know of any alternative to update backdrop core from the command line please?

You may be able to do this with brush - this tool is no longer supported but I believe it has this command and it may work. But be aware that you will not get any support if it doesn't. Alternatively, you could script how you're currently doing it.

However, be encouraged that your interest in this ticket has spurred me on, so hopefully we won't have too long to wait for this.

Also, where do you recommend I look around to try and help you out if I can?

It's great that you're interested in helping. Have a look at the issue queue and see if you fancy doing a PR for any of the tickets there. If you need any pointers just ask.

TheMetMan commented 1 year ago

@marioawad I am away from home until tomorrow, I will contact you again then with my scrips to update core and modules using bee with my mods.

yorkshire-pudding commented 1 year ago

Hi @TheMetMan - I have created a new branch as I think this may require a bit of work before merging to 1.x-1.x . https://github.com/backdrop-contrib/bee/tree/issue-111-update-command

Please rebase (Sync) your repo to the latest bee and then do a PR to this branch rather than to the 1.x-1.x branch. Don't worry if you create the PR to the wrong branch as I can edit this to the right branch. I will then create some suggestions against your PR.

TheMetMan commented 1 year ago

@yorkshire-pudding , I am not sure how to do this. I know I should know, but I do not. I have read the docs, and still not sure. Any chance of a chat about it somewhere off this Issue? I have a video call system with my Email provider. If we could agree a time, I could send you a link. Regards

yorkshire-pudding commented 1 year ago

Hi @TheMetMan - of course. I was new to this a year ago and I'd be happy to show you. It might be a bit complex as your bee repo is not a fork, but there are ways we can work around that. I'm relatively flexible during the week, less so at the weekend. Not sure the best way to share details with you as you're not on Zulip. You could contact me through my website.

TheMetMan commented 1 year ago

@marioawad > Do you know of any alternative to update backdrop core from the command line please? I have created some scripts to make upgrading Backdrop easy from the command line. The link takes you to a method of creating a Backdrop site very easily. Once done there is a script called runBackdropUpgrade which needs the other scripts to run correctly. You will need the version of Bee which you will find here in particular the issue-111-update-command branch for it to work. This is work in progress by the way but it does work. Also I have created some Ansible Scripts to upddate multiple Backdrop Websites if that is of interest.

marioawad commented 1 year ago

@TheMetMan Thanks for sharing all the info and links and since I'm new to the backdrop ecosystem it looks like I have plenty to dive into which I'll certainly enjoy. Thanks again to both of you and have a great day!

leeksoup commented 7 months ago

What is the status of this, please? This is the number 1 thing that I used drush / brush for until they stopped working ...

yorkshire-pudding commented 7 months ago

Hi @leeksoup - there is something there that works reasonably well on single site installations (i.e. not multi-site) and @TheMetMan, who started work on this feature is using regularly. It is on this branch.

Depending how you have bee installed it may simply be a case of doing git checkout issue-111-update-command I am merging changes made in the main branch into this feature branch so if you do decide to use that branch you won't miss out on any other features in bee.

There is a warning at the top to not use on production, so if you use it and it breaks your site, then it is at your own risk; the risk is that running an update deletes a module folder and then the download from GitHub fails for some reason and you are left without a critical module until you can repair it.

Please also note, that it does not yet update core.

After a while with very little love, it is now getting some attention again and one bug and one enhancement have been merged into the feature branch thanks to @hosef. Alas, the need to earn a living does limit the time I can spend on it, so it is not progressing as fast as we would like. It is good to know there is interest though and it does help to motivate developers.

leeksoup commented 7 months ago

@yorkshire-pudding - thank you for the detailed response. I was in a hurry so I ended up semi-manually updating by* removing the directories for core and the modules that needed to be updated, and then downloading with bee, restoring / merging settings.php and htaccess, and then updating the DB, also with bee.

Next time, I will try this feature branch. I completely understand about not using on a production site. My workflow is generally update on dev site -> test -> deploy to production.

Oh, for sure there is interest in this feature! I'm sure I'm not the only one. It would probably help point users to his helpful package if "how to install using bee" (and when it is possible, "how to update using bee") were added to the Backdrop Documentation.

ETA: * first making backups, and then ...

leeksoup commented 5 months ago

So I'm looking to update again and wondering if the status has changed or if @yorkshire-pudding's last comment is still the way to go.

yorkshire-pudding commented 5 months ago

@leeksoup - I'm afraid the advice remains the same. I have a clearer idea of what is needed, but, alas, the need to earn a living is preventing me spending the time I would like on this feature

leeksoup commented 5 months ago

Understood, @yorkshire-pudding!

Turns out I only have Backdrop itself to update this time ... ah well.

manu-mei-singh commented 5 months ago

Thanks so much @yorkshire-pudding and @TheMetMan for all your work on this ! I and ,I am sure others really appreciate it. The command bee ups works great! I am wondering if there is a road map for that command , bee ups, to also list the backdrop core, something like brush -n up. Or maybe it already does and I missed something? I also understand your needs about earning a living. I am happy to help out as my team (Palante Technology Cooperative) is very interested in this working.

Again, thanks for all your hard work in making this all happen.

Manu

yorkshire-pudding commented 5 months ago

Hi @manu-mei-singh . Thank you for your interest and the offer of help; Palante have been good friends of Backdrop over the years.

I do have a plan to add support for core updates. This is in #296 . However, I believe I'm currently blocking this, as quite a few things are now depending on #348 which will make the download_bee_download_project() function more flexible for other functions and help to fix some of the bugs. I have an idea of how to do this, but I haven't been able to devote any time to this recently.

If any of your team is interested in getting involved, I would be happy to discuss as it wouldn't be good to duplicate any effort.

manu-mei-singh commented 4 months ago

hi @yorkshire-pudding , I can devote some time to getting involved later this month or in early march. Let me know how you would like to proceed.

thanks!

yorkshire-pudding commented 4 months ago

hi @yorkshire-pudding , I can devote some time to getting involved later this month or in early march. Let me know how you would like to proceed.

thanks!

Hi @manu-mei-singh - please have a look and see if there are any of the tickets that you would be confident in tackling. I can't tell from your GitHub profile how much PHP work you've done so don't know what your ability level is.

manu-mei-singh commented 4 months ago

@yorkshire-pudding , I am in the process of looking around at the tickets. unfortunately It will be slower going than I had planned...

leeksoup commented 2 months ago

@yorkshire-pudding - I just tried the bee update functionality with the somewhat-recent Entity Plus update and it appears to have worked just fine.

herbdool commented 1 month ago

@yorkshire-pudding first time I've taken a look at this. My initial thought is that putting a project update command in update.bee.inc is the wrong place for it, in my not-so-humble opinion. That file is about running update hooks (for database/config changes) and not about updating a codebase. So I think putting it in download.bee.inc makes more sense, since it's really a more sophisticated version of that, eliminating the need to delete the project code before downloading.

yorkshire-pudding commented 1 month ago

@herbdool - thanks for your thoughts.

My initial thought is that putting a project update command in update.bee.inc is the wrong place for it, in my not-so-humble opinion. That file is about running update hooks (for database/config changes) and not about updating a codebase.

My former co-maintainer and I disagree. The file, according to the file doc, is actually about:

/**
 * @file
 * Command(s) for updating Backdrop projects.
 */

So I think putting it in download.bee.inc makes more sense, since it's really a more sophisticated version of that, eliminating the need to delete the project code before downloading.

This will use an enhanced version of download - see #348 - but it doesn't need to be in the same file, and from my point of view it is more logical in update, as well as preferable in keeping download focussed in scope. Update hooks are typically associated with updating the code so they do, in my opinion, and from the user interface process, belong together.