Consider how to move test configuration to live

[andrewvanbreda]

Consider how to move test configuration that has been setup to live environment

[andrewvanbreda]

@BirenRathod Hi Biren, Do you have any thoughts how we will move the configuration of the test site onto live when finished, as I think there is a live site already isn't there with holding page so presumably the usual Pantheon thing where is it activated and moved across won't be happening.

Thanks for any thoughts you may have regarding this.

[BirenRathod]

@andrewvanbreda All stuff from test to live already moved, I just have to update this week changes from test to live.

[andrewvanbreda]

@BirenRathod @kitenetter When was that move done?, as obviously it means manual moving of all configuration across, including translations, all Martins stuff. Not sure if that is feasible. How out of date is live? Could you let me know the URL for it also.

[BirenRathod]

@andrewvanbreda I moved about 15 days ago. it is ukpoms.org.uk. Don't worry about how old stuff on it because once we finish testing and decide to make site live we will move everything again. it is not a huge process.

[andrewvanbreda]

@BirenRathod Hi Biren, Am a bit confused what is happening. What is moving when you say "we will move everything again"? I thought the configuration won't be moved from test to live again and that is why there is a need to set live up manually. Thanks for any clarification.

[BirenRathod]

@andrewvanbreda Which configuration are you after?

[andrewvanbreda]

@BirenRathod Drupal database configurations, like the Indicia page setups on the test site. Anything in the Drupal database really.

[DavidRoy]

The test website points to the live warehouse. I assume we'll replace the current live website with the test website when everyone is happy with it?

[BirenRathod]

@andrewvanbreda Test website is already link to live, so I can move everything to live.

[andrewvanbreda]

@DavidRoy @BirenRathod Ok thanks, that is what I was checking on, that it can be moved across when ready. Great thanks.

[andrewvanbreda]

@DavidRoy @kitenetter @CCarvell @BirenRathod Personally I think best thing to do today is we should move the site onto live as soon as you are ready with the parts you are doing today. I can continue to work and make changes in parallel on test/live at same time. It doesn't make too much difference to me for a few reasons (e.g. makes no difference to report work, work can be done on test and easily copied to live for me)

[CCarvell]

@andrewvanbreda @kitenetter I am flagging this here given it's the issue thread you are following today and probably links to report work you may still be doing - so apologies if not relevant. BUT on the Map of FIT Counts page, the Table below the map is still displaying counts from March 2021 which look like the dummy counts (pics of animals etc!). https://test-ceh-poms.pantheonsite.io/fit-counts-map Can you see my screenshot below? Can we strip out these dummy counts so that only real counts from April onwards are displaying?

[andrewvanbreda]

@CCarvell Dummy data needs to be deleted ideally. I don't have an exact definition of what constitutes something that is dummy, I don't want to be deleting real data. Are you able to send me the numbers of ones to delete.

[CCarvell]

@andrewvanbreda I would suggest deleting all counts dated March 2021. There are also - I think - two counts from 02/04 on Bramble and Buddleia that don't look right as those are not in flower yet. I cannot now view the table as the pantheon site isn't showing it. From memory the first April count from 2021 should be showing as being on Dandelion.

Will this affect the live quote at the top of the page "166 counts submitted so far this year by 55 participants"??

Also will this just be deleting from the counts on display or will it be deleting from the whole database/ warehouse? @kitenetter may think of a reason to retain these dummy counts in our records?

[andrewvanbreda]

@CCarvell @kitenetter Yes that value will change as the system doesn't know what is fake.

Yes this would affect everywhere including the warehouse. However the values will not literally be deleted, there is a flag that is set on the data saying "deleted=true". So they can be undeleted providing we have and id number or something to identify them with.

[CCarvell]

@andrewvanbreda thanks - I think you can go ahead and delete those March etc counts now.

[kitenetter]

@andrewvanbreda re deletions: In #77 we said to remove:

• All occurrences from survey ID 641 (FIT Count app) where sample ID <= 14202516

I'm checking the other surveys and will let you have the IDs to delete shortly.

[andrewvanbreda]

These are the ones I counted from march, let me know if you think this is right

13803554,13845947,13959723,13955682,13960608, 13959849,13959785,13960126,13960019,13959920,13959917,13960760, 13961262,13959859,13963166,13963134,13963084,13963079,13963074, 13963062,13963249,13963248,13963134,13963084,13963079,13963420, 13963166,13963062,13963074,13968594,13968590,13987347,13983906, 13987300,13987253,13987252,13987248,13987247,13987216,13987192, 13987252,13987248,13987247,13987216,13987185,13988296,13992134, 14041612,14021694,14023621,14041648,14039877

[andrewvanbreda]

@kitenetter Your statement includes some April ones too, so let me know if those need to go also, the list I provided is march only

[kitenetter]

For survey ID 636 (PoMS FIT Count (website)) there appear to be just four samples, all of which can be deleted.

For survey ID 637 (PoMS FIT Count 1km (website) there appears to be just one sample, which can be deleted.

For survey ID 641, I agree that the March samples you list can be deleted, and these from April are also to be deleted: 14068116, 14069021, 14097873, 14144766, 14181012, 14196022, 14202516, 14258170

There are a few others where it is not clear if they were dummy ones or real - I will check up on these but they can remain for now.

[andrewvanbreda]

@kitenetter Ok, Some of that is my data, and actually it is useful for me to leave some app data on site, so let's delete your selections last minute and delete the other list for now. That is clear then that your suggestions aren't done yet and March ones are.

[andrewvanbreda]

@BirenRathod If the first statement correctly updates 52 rows, then can you run the other 2 also. If not 52 rows updated on first statement, do not run second two, as that means there is a problem with the first test statement

update indicia.samples set deleted=deleted where id in (13803554,13845947,13959723,13955682,13960608, 13959849,13959785,13960126,13960019,13959920,13959917,13960760, 13961262,13959859,13963166,13963134,13963084,13963079,13963074, 13963062,13963249,13963248,13963134,13963084,13963079,13963420, 13963166,13963062,13963074,13968594,13968590,13987347,13983906, 13987300,13987253,13987252,13987248,13987247,13987216,13987192, 13987252,13987248,13987247,13987216,13987185,13988296,13992134, 14041612,14021694,14023621,14041648,14039877);

update indicia.samples set deleted=true,updated_on=now(),updated_by_id=5553 where id in (13803554,13845947,13959723,13955682,13960608, 13959849,13959785,13960126,13960019,13959920,13959917,13960760, 13961262,13959859,13963166,13963134,13963084,13963079,13963074, 13963062,13963249,13963248,13963134,13963084,13963079,13963420, 13963166,13963062,13963074,13968594,13968590,13987347,13983906, 13987300,13987253,13987252,13987248,13987247,13987216,13987192, 13987252,13987248,13987247,13987216,13987185,13988296,13992134, 14041612,14021694,14023621,14041648,14039877);

insert into indicia.work_queue(task, entity, record_id, cost_estimate, priority, created_on) select 'task_cache_builder_update', 'sample', id, 100, 2, now() from indicia.samples where id in (13803554,13845947,13959723,13955682,13960608, 13959849,13959785,13960126,13960019,13959920,13959917,13960760, 13961262,13959859,13963166,13963134,13963084,13963079,13963074, 13963062,13963249,13963248,13963134,13963084,13963079,13963420, 13963166,13963062,13963074,13968594,13968590,13987347,13983906, 13987300,13987253,13987252,13987248,13987247,13987216,13987192, 13987252,13987248,13987247,13987216,13987185,13988296,13992134, 14041612,14021694,14023621,14041648,14039877);

[BirenRathod]

@andrewvanbreda First query has updated 42 rows. Is that correct?

[BirenRathod]

@andrewvanbreda I have attached the select query result of first your first update query. data-1621437045968_samples.zip

[andrewvanbreda]

@BirenRathod Thanks Biren, that is strange but useful. I will check

[andrewvanbreda]

@BirenRathod Hold fire on this for now. Something odd here.

[andrewvanbreda]

All test data removed manually, including my test data and Martin's suggestions

[andrewvanbreda]

@kitenetter @CCarvell Just noticed there are March records not in the ID lists we talked about so i will remove those too.

[andrewvanbreda]

Have removed following extra March records and also some April ones that looked wrong as they had Playstation controllers and bits of sofa showing

13959723 13960294 13963394 13963412 13963422 13987787 13987868 13987886 13987887 14119810 14119811 14119812 14143673

[andrewvanbreda]

Site is now live