search

BiologicalRecordsCentre / iRecord

Repository to store and track enhancements, issues and tasks regarding the iRecord website.
http://irecord.org.uk
2 stars 1 forks source link

iRecord Drupal 9 website #980

Closed kitenetter closed 3 years ago

kitenetter commented 3 years ago

Following discussions this morning we agreed that it would be sensible to explore implementing a Drupal 9 version of iRecord, since the step from 8 to 9 is a small one. Has implications for related libraries and PHP.

@BirenRathod to investigate using iForms in Drupal 9.

BirenRathod commented 3 years ago

@johnvanbreda As I mentioned, I have installed the module but it does produces some error which is expected and suggests, it does require implementation to work with Drupal 9.

Drupal\Component\Serialization\Exception\InvalidDataTypeException: drupal-9/modules/custom/iform/iform.routing.yml: Duplicate key "iform.joingroup" detected at line 53 (near " _permission: 'access content'"). in Drupal\Core\Discovery\YamlDiscovery->decode() (line 24 of drupal-9\core\lib\Drupal\Core\Discovery\YamlDiscovery.php).

johnvanbreda commented 3 years ago

I'm about to try another D9 installation so will get back to you on this.

BirenRathod commented 3 years ago

I also found this link from Pantheon which is useful to implement D8 code to D9. Not many changes required. https://pantheon.io/docs/drupal-9#can-i-upgrade-my-existing-drupal-site-to-drupal-9

johnvanbreda commented 3 years ago

It turned out to be quite a lot of work as I had to convert lots of jQuery UI code which is no longer easily supported. I've now managed to get it working though haven't tested all functionality.

To use Drupal 9 you need:

The feature-d9 branches can all be merged back into the main branches once tested as the changes are all compatible with D8 & D9.

There are lots of changes in the above branches which will need testing:

Additionally:

Note that I've removed deprecated Drupal 8 code from all the core stuff, but not project specific forms since I don't want to introduce bugs that I can't easily test for, so they will need some work when migrating sites to Drupal 9 (e.g. NPMS, UKBMS).

Over to you @BirenRathod !

BirenRathod commented 3 years ago

Thanks for @johnvanbreda. Really appreciated. I will test on my machine and let you know.

I think project specific should more concern to Andy & Gary as they need to test & make sure its functionality.

BirenRathod commented 3 years ago

@johnvanbreda This is my initial assessments about iForm Drupal 9. Here is the URL - > http://test.brc.ac.uk/irecord9/

BirenRathod commented 3 years ago

@johnvanbreda I have added the reference in libraries for missing jquery datepicker and it works fine now. Also, it resolved the layout issue too.

BirenRathod commented 3 years ago

@johnvanbreda Some more....

  1. jQuery error from taxa_search.php although it did get list of species.
  2. Can't upload photo. below is the error message from jquery. $(...).progressbar is not a function
  3. Without photo record submitted fine.
johnvanbreda commented 3 years ago

We are trying to move away from jQuery.ui so adding the reference to the ui.datepicker library is not the right approach. Looking at the code, the old datepicker is still referenced from a few project specific forms which will need to be replaced. I noticed it was also referenced from the lock icon code which I missed so converting that to support the HTML 5 date input is a task to do. Are you sure you are using feature-d9 on the media library? It should use an HTML5 progress element now rather than jQuery UI.

BirenRathod commented 3 years ago

@johnvanbreda I know you don't want to use jquery datepicker but I seen other references of jquery and It will allow me to test further, so I added but it is in feature-d9, so you can just ignore when you merge the code in the master. i didn't as in library in media repository it is just referenced in the iform.libraries.yml and install the datepicker module.

BirenRathod commented 3 years ago

@johnvanbreda yes I'm sure it is using feature-d9 media library.

BirenRathod commented 3 years ago

@johnvanbreda Sorry my mistake, I checked media & client_helpers folder again and founf the settings was not right. So no amendment needed for date picker.

But photo upload problem still there. it is not showing jquery but just html error & also error from taxa_search.php file.

BirenRathod commented 3 years ago

@johnvanbreda Although my mistake of adding datepicker jquery has solved most jquery problems. That was not expected. After I removed datepicker jquery, now tab is not working and throws error. so that is another one to look.

BirenRathod commented 3 years ago

I think you need to remove the jquery datepicker validator and add individual validator for HTML 5 date.

johnvanbreda commented 3 years ago

@BirenRathod please can you post details of the jQuery.ui.datepicker error as well as the one in taxa_search.php as I'm not seeing those problems.

BirenRathod commented 3 years ago

@johnvanbreda you can see on the test site which I setup here test.brc.ac.uk/irecord9

BirenRathod commented 3 years ago

@johnvanbreda you will error in debug panel not on the page.

BirenRathod commented 3 years ago

@johnvanbreda if you can't find below are the errors. Ignore my message about taxa_search.php that may be not clearing cache.

  1. Uncaught TypeError: indiciaData.dateFormat is undefined
  2. Uncaught TypeError: $.validator.methods[method] is undefined
  3. Photo upload html error.
johnvanbreda commented 3 years ago

I think the first 2 errors are fixed if you pull the latest code. For error 3, this works on my local machine but you are getting ERR_CONNECTION_RESET. You will need to debug whether the request is reaching upload.php or not.

BirenRathod commented 3 years ago

@johnvanbreda Thanks for those changes you made. After that, I installed iform on my machine and it works fine. Then I pulled the code on test but it was not working. I uninstalled it and reinstall everything and it works fine. I think that was due to not clearing cookies on browser.

Just small issue I found but this is not urgent. When I uninstalled and reinstalled iform & easy login module, it didn't uninstall completely all fields from the database even though there are no values stored in the database. I know this problem already exists in Drupal 8.

BirenRathod commented 3 years ago

also photo upload doesn't work if site setup as subsite which is why it didn't work on test.brc.ac.uk/irecord.

BirenRathod commented 3 years ago

@johnvanbreda & @DavidRoy Everything works fine, so we could go ahead with Drupal 9 for iRecord.

@johnvanbreda If you can pass me some more info about indicia block, would be good.

johnvanbreda commented 3 years ago

Please pull the feature-d9 branches for client_helpers and the indicia_features folders. The Indicia blocks module just gives you 2 simple blocks to add to the home page - one with a list of records, the other with a map. You can set up a home page as a basic page, then use the Layout Builder module. Set the Display settings for the Basic Page content type to use Layout Builder and to allow each page to have it's own layout. Then your home page will have a "Layout" tab that lets you create a layout and add blocks.

BirenRathod commented 3 years ago

@johnvanbreda Thanks for this details. I already pulled those branches when I pulled the iform. I can see those two blocks in the blocks list too and I'm aware of layout settings too. My question was regarding configuration in the block, like using URL or anything... but from your above details, it should be straight setup. So I will give it a go.

BirenRathod commented 3 years ago

@johnvanbreda, @DavidRoy, @kitenetter & @burkmarr

I have now migrated the irecord Drupal 7 to Drupal 9, below is the link. http://test.brc.ac.uk/irecord9/

@johnvanbreda - There is no verification 3 or 4 in Drupal 9, so I replaced them with verification 5. I haven't tested yet but that I will do next.

If you find any problem or missing let me know. I will make above changes in couple of days but it is ready to test. You can submit record in training mode as it is link to live warehouse.

johnvanbreda commented 3 years ago

@BirenRathod thanks. Which pages required verification 3 or 4? I thought that we've been using Verification 5 or the new Elasticsearch code for a long while now.

Apart from the missing home page, when I log in as myself, the menu has lots of missing stuff, e.g. the links to the standard enter a list of records or enter a casual records forms. If I reconstruct the expected URL for the enter a casual record form then I get an access denied.

I noticed that the HTML5 date control doesn't have the form-control class so looks wrong on Bootstrap - I will fix this in the code.

I also found that the Edit page for my account has lots of bits missing, I can't select my location, taxon groups, or enable vague date entry for example.

BirenRathod commented 3 years ago

@johnvanbreda If you look at nodes -> 9, 470, 3186 have got verification 3 & 470 has got verification 4.

some of links in menu need to sort out the permission bit, which I will do soon.

Yes, I have hidden for time being on "My account", which I will sort out soon too.

johnvanbreda commented 3 years ago

Here's the commit to add the class to the date picker for Bootstrap: https://github.com/Indicia-Team/client_helpers/commit/8dd5288b19f569c9f51f290f3df2049ae48eb3d0

Node 470 is old, hopefully unused. I think it can be deleted.

I could not find a node 3186, but 3188 is the current "old verification" page on the menu and that is using verification 5.

BirenRathod commented 3 years ago

@johnvanbreda

johnvanbreda commented 3 years ago

Thanks - that looks better.

BirenRathod commented 3 years ago

@johnvanbreda

0 D:\websites\drupal9\modules\custom\iform\client_helpers\prebuilt_forms\includes\dynamic.php(694): extension_misc_extensions::breadcrumb(Array, Array, NULL, Array, '/irecord9/moors...', Array)

DavidRoy commented 3 years ago

@BirenRathod

BirenRathod commented 3 years ago

@DavidRoy

All issues have been resolved now except the last one which needs to change the code in theme file. I will let you about it soon.

johnvanbreda commented 3 years ago

@BirenRathod some more issues for you:

BirenRathod commented 3 years ago

@johnvanbreda

BirenRathod commented 3 years ago

@johnvanbreda

BirenRathod commented 3 years ago

@johnvanbreda more about permission... If I don't enabled the 'authenticated user' permission of 'Access Indicia Iform web pages' then you will see error of elastic search and don't see any record & can't get access of data entry form & taxon based entry form which David has mentioned above.

johnvanbreda commented 3 years ago

"Access Indicia Iform web pages" is the main permission required for any Indicia forms and reports so should be granted to anonymouse and authenticated users. There is a similar "Access iform" permission which is caused by the migration of pages which refer to this permission by name in their settings. As long as they both have the same settings it won't matter.

I rebuilt the content permissions which fixed a few things.

I also removed the Masquerade block from the bottom of all pages except home, since the autocomplete clashes with ours.

Some more problems - the login and logout system seems broken. For example, if I'm logged in as admin, then I log out, when I try to log in again it is still logged in as admin. The logout link doesn't work properly. Also, if I am logged in, then the home page still has a login link in the menu. I think it is basically broken because the links are toggling me between irecord and irecord9 in the URL - until this is sorted I don't think I can work on the other issues.

BirenRathod commented 3 years ago

@johnvanbreda Both "Access Indicia Iform web pages" & "Access iForm" have got same permissions.

DavidRoy commented 3 years ago

@BirenRathod the site is giving a '404 - File or directory not found.' error for me today http://test.brc.ac.uk/irecord9/

BirenRathod commented 3 years ago

@DavidRoy Yes, because URL has changed now. It is now test.brc.ac.uk/irecord

johnvanbreda commented 3 years ago

I think there were some cached links, which caused the problems with login/logout - they seem to have gone now.

On the Explore My Records page, at the top of the form structure there is supposed to be the following:

<input type="hidden" class="es-filter-param" data-es-bool-clause="must" data-es-field="metadata.created_by_id" data-es-query-type="term" value="{{ indicia_user_id }}" />

This is the hidden input that filters all requests to the logged in user's data. But, on the Drupal 9 development site it looks like this:

<input type="hidden" class="es-filter-param" data-es-bool-clause="must" data-es-field="metadata.created_by_id" data-es-query-type="term" value="" />

The token {{ indicia_user_id }} is missing. If I edit the page and add it back, then save it, the save does not "take" and the page is still unfiltered. Something is stripping this token out. I have a local D8 test setup and a local D9 test setup and I can reproduce this problem on Drupal 9 but not Drupal 8 so it must be new. I guess it relates to the token looking like a twig replacement.

johnvanbreda commented 3 years ago

I've managed to confirm the problem with {{ indicia_user_id }} is not saving, as it is correct in the database, but loading. I'm looking into this now.

johnvanbreda commented 3 years ago

@BirenRathod I've just pushed an update to the iform module develop branch which fixes the problem I found above.

BirenRathod commented 3 years ago

@johnvanbreda I updated the code from develop branch. Clear cache and reload the 'My records' page, don't see any impact. Am I missing anything?

BirenRathod commented 3 years ago

@johnvanbreda After I added {{ indicia_user_id }}, it works nicely.

johnvanbreda commented 3 years ago

I think the {{ indicia_user_id }} value needed to be resaved into the code - it's now working though my personal My Records page is showing an odd set of records which I'm looking into.

johnvanbreda commented 3 years ago

Minor thing - https://www.brc.ac.uk/irecord/explore/nhm-wlg shows a full width report, the equivalent on the dev site is narrow.

Activity summaries seem to be broken - http://test.brc.ac.uk/irecord/simple-activity-summary?group_id=1325&implicit=f

Also, I don't know why but on my user account, My Records shows > 4000 records for iRecord live, but only 80 records for iRecord dev site of which some aren't mine. The request looks the same though (using browser dev tools and the Network tab) so I can't see why, as they are pointing to the same server.

BirenRathod commented 3 years ago

@johnvanbreda

This page http://test.brc.ac.uk/irecord/simple-activity-summary?group_id=1325&implicit=f throws error, like this one Error: Call to undefined function variable_get() in irecord_preprocess_page() (line 30 of D:\websites\drupal9\themes\custom\irecord\irecord.theme)

basically it needs a value of indicia_website_id and indicia_password.

johnvanbreda commented 3 years ago

Presumably the theme is not on Github so you will have to fix this yourself. If you look elsewhere for calls to iform_get_connection_details() you will see how to do it.