Opening dataview from profiles and restyling the intake profile visualization

[JacobBirk]

Now the intake at wells are colored according to the selected chemical (hopefully - it is difficult to se on the demo:-)) We would like to add an event to "click on intake" that open the Datamenu for the current borehole.

[sashuk]

@mapcentia @JacobBirk Implemented in https://github.com/sashuk/watsonc/commit/803bd2b942a790ef50fa6ab9736e4d4632834c11

Demo:

[JacobBirk]

Very nice. The intakes should be colored green, yellow and red bases on latest value.

Looks like the color scheme for the layers in the wells are not correct.

[sashuk]

@JacobBirk Could you please clarify if intake colors should display only the latest value, not the maximum and latest?

[JacobBirk]

I think that it might be diffucult to have both colors at the intake, even though ud would be nice. You are welcome to come with a suggestion.

[sashuk]

@JacobBirk What if show the latest and maximum colors under the actual chemical name and intake number like in the example below? Should I also check the Python script concerning the colors in wells (the colors in wells are defined in the Python script and are returned from the Python script with the profile data)?

[JacobBirk]

I think that the colors somehow should be conneced to the intake squared boxes.

Where did you think that the box you suggest should be located?

Yes please check i python script. We might need to have Ole involved, if you can’t see the problem.

Jacob

---

Fra: Aleksandr Shumilov notifications@github.com Sendt: Tuesday, July 23, 2019 12:25:55 AM Til: sashuk/watsonc watsonc@noreply.github.com Cc: Jacob Birk Jensen jbj@watsonc.dk; Mention mention@noreply.github.com Emne: Re: [sashuk/watsonc] Opening dataview from profiles (#67)

@JacobBirkhttps://github.com/JacobBirk What if show the latest and maximum colors under the actual chemical name and intake number like in the example below? Should I also check the Python script concerning the colors in wells (the colors in wells are defined in the Python script and are returned from the Python script with the profile data)?

[intake]https://user-images.githubusercontent.com/1390758/61669465-b6d5eb00-ad12-11e9-9fc9-eb90761e7b24.PNG

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/sashuk/watsonc/issues/67?email_source=notifications&email_token=AL7IMDNMZAGQ3VTRIPXEZR3QAYXXHA5CNFSM4IFNU5RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2RK7SA#issuecomment-513978312, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL7IMDMTUTIQRAHJSMXSTLDQAYXXHANCNFSM4IFNU5RA.

[sashuk]

Got it, thanks! As far as I understand the UI checks where the measurement value is located in relation to the chemical limits, so it is either green, orange or red. However, the Python script uses a completely different color assignment, so I will check how it works then!

Concerning the box - it can be located right under the move chemical symbol and the chemical name, so it will have more space than now and will be more visible.

[JacobBirk]

Ok, could you make an example on a profile?

Jacob

---

Fra: Aleksandr Shumilov notifications@github.com Sendt: Tuesday, July 23, 2019 9:02:59 AM Til: sashuk/watsonc watsonc@noreply.github.com Cc: Jacob Birk Jensen jbj@watsonc.dk; Mention mention@noreply.github.com Emne: Re: [sashuk/watsonc] Opening dataview from profiles (#67)

Got it, thanks! As far as I understand the UI checks where the measurement value is located in relation to the chemical limits, so it is either green, orange or red. However, the Python script uses a completely different color assignment, so I will check how it works then!

Concerning the box - it can be located right under the move chemical symbol and the chemical name, so it will have more space than now and will be more visible.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/sashuk/watsonc/issues/67?email_source=notifications&email_token=AL7IMDKJUENXZOVOPBLGJJLQA2UKHA5CNFSM4IFNU5RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2SEESA#issuecomment-514081352, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL7IMDLK363POYKOMZF4QZDQA2UKHANCNFSM4IFNU5RA.

[sashuk]

@JacobBirk Sorry, I was meaning the measurement chemicals panel (plots are constructed from these measurements). However, should the intake colors in image A correspond to the intake square box in the profile from image B?

Image A:

Image B:

The Python source takes colors from the CSV file https://github.com/WatsonC-hub/profil/blob/master/RockSymbol_color_code.csv, so probably colors should not be identical?

[JacobBirk]

Yes exactly. The color of the intake must correspond to the color of the circle on the image A. The question is whether there can be one or two colored on each intake. I think we should settle for a color similar to the last measurement, unless you see another solution

Jacob

---

Fra: Aleksandr Shumilov notifications@github.com Sendt: Tuesday, July 23, 2019 10:01:42 AM Til: sashuk/watsonc watsonc@noreply.github.com Cc: Jacob Birk Jensen jbj@watsonc.dk; Mention mention@noreply.github.com Emne: Re: [sashuk/watsonc] Opening dataview from profiles (#67)

@JacobBirkhttps://github.com/JacobBirk Sorry, I was meaning the measurement chemicals panel (plots are constructed from these measurements). However, the intake colors in image A should correspond to the intake square box in the profile from image B?

Image A: [a]https://user-images.githubusercontent.com/1390758/61693581-6c7c5a80-ad62-11e9-8c04-5018b84fae59.PNG

Image B: [b]https://user-images.githubusercontent.com/1390758/61693638-85850b80-ad62-11e9-9c32-0be2f4595365.PNG

The Python source takes colors from the CSV file https://github.com/WatsonC-hub/profil/blob/master/RockSymbol_color_code.csv, so probably colors should not be identical?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/sashuk/watsonc/issues/67?email_source=notifications&email_token=AL7IMDM2C2EFT73K3KFFZOTQA23GNA5CNFSM4IFNU5RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2SISWY#issuecomment-514099547, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL7IMDORNTCUDI4MUQMRQWDQA23GNANCNFSM4IFNU5RA.

[sashuk]

@JacobBirk Assuming that intake colors in Image A should correspond to the ones in profiles, then it means the entire UI should not use color codes and categories from codes.compunds table, but to use color codes from RockSymbol_color_code.csv?

Right now the UI requests the codes.compunds table and receives the categories and chemicals limits and other meta. According to received chemicals meta, the coloring is performed using only three colors - green, orange, and red. However, the profiles script uses more complex coloring scheme, which requires additional data that is not available in the Vidi now.

Also could you please specify if The question is whether there can be one or two colored on each intake is about the Image A or Image B coloring? In the case of Image A I would keep it the current way, as it shows both latest and maximum values.

[JacobBirk]

@sashuk we are only talking about changes in the profile view and only the colors of the intake box shown with dark blue in image B. Everything else in the profile should follow the specified color scheme. Sorry for the confusion. Please ask again if the answer isn't good enough

[sashuk]

@JacobBirk Thank you very much! So, the color of the intake box should be green, orange or red, just like in Image A? For displaying both latest and maximum value I propose to implement it like this (the latest value is the upper one, the maximum value is the lower one):

[JacobBirk]

Perfect. Let’s try this😉

Jacob

---

Fra: Aleksandr Shumilov notifications@github.com Sendt: Wednesday, July 24, 2019 1:15:19 PM Til: sashuk/watsonc watsonc@noreply.github.com Cc: Jacob Birk Jensen jbj@watsonc.dk; Mention mention@noreply.github.com Emne: Re: [sashuk/watsonc] Opening dataview from profiles (#67)

@JacobBirkhttps://github.com/JacobBirk Thank you very much! So, the color of the intake box should be green, orange or red, just like in Image A? For displaying both latest and maximum value I propose to implement it like this (the latest value is the upper one, the maximum value is the lower one):

[coloring]https://user-images.githubusercontent.com/1390758/61789470-598f8600-ae47-11e9-9aff-92fabb7ed1f9.PNG

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/sashuk/watsonc/issues/67?email_source=notifications&email_token=AL7IMDJKPXZORDHD4SWGPU3QBA2UPA5CNFSM4IFNU5RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2WAK4Q#issuecomment-514590066, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL7IMDOXOTSONKATRCWHPCDQBA2UPANCNFSM4IFNU5RA.

[sashuk]

@JacobBirk Great, I will get to Python script modification then! Thank you, however, I think it would be still good to know why the more complex system was used originally from @olemunch

[JacobBirk]

As I understand the complex system from @olemunch is for laver coloring and coloring of layers for each borehole. The only adjustments I suggest is for the coloring of the intake boxes. Agree?

[sashuk]

Sure!

On Wed, 24 Jul 2019 at 23:56, JacobBirk notifications@github.com wrote:

As I understand the complex system from @olemunch https://github.com/olemunch is for laver coloring and coloring of layers for each borehole. The only adjustments I suggest is for the coloring of the intake boxes. Agree?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sashuk/watsonc/issues/67?email_source=notifications&email_token=AAKTRJUEOGGKZKX5U6RV6D3QBB3UXA5CNFSM4IFNU5RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2WZK7Q#issuecomment-514692478, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKTRJUG7PR5PP727DDSMCLQBB3UXANCNFSM4IFNU5RA .

--

---

Aleksandr Shumilov

Tel: 79501017750 Skype: sashuk-morehod Email: alexshumilov@yahoo.com Web-site: https://alexshumilov.ru

[olemunch]

Hi @sashuk I'm back from vecation :-)

If I can help clarify details in the profile tool let me know.

[sashuk]

@olemunch Hello, Ole! Great, thank you!

[sashuk]

@olemunch Could you please specify where the color of the intake bars is defined? I've changed the https://github.com/WatsonC-hub/profil/blob/master/profiletool_kemi_v5.py#L542, so the three gradation colors correspond to the colors that are used in the UI (as was discussed before - the colors in intake bars of the profile should correspond to the ones in the UI). However, some of the intake bars are still having colors, different from those three ones. Please see the example below:

The profile for Nitrat is built. The intake bar is colored in a grey color. According to the code the only place where intake bar color is defined is the https://github.com/WatsonC-hub/profil/blob/master/profiletool_kemi_v5.py#L559 (which can take only three possible colors), but the color is still different. Could you please specify how else the intake bar color can be defined?

[WatsonC-hub]

Hi @sashuk Firstly I did not create this par of the code - but your right that is the correct place.

I think I know what the problem is. At the moment we draw layers and boreholes on two different axis to avoid a problem in plotly with colors being blended with each other. Nnow I think the color of the screen (intake bar) i beeing affectted by the color on the borehole behind.

One solution could be to create yet another y-axis for the intake-bars.

A test could be to add eg 10 m to the x-location line 549 placing the bars outside the boreholes. Colors should then not be affected by other colors.

If you cannot solve this easily I can have Niels look at it on monday

/Ole

[sashuk]

@WatsonC-hub @olemunch The test was performed, seems like colors stayed the same:

Modifications:

if last_chem > data_limit['limit'][0]:
    farve ='rgb(252, 60, 60)'
elif last_chem > data_limit['attention'][0]:
    farve = 'rgb(247, 168, 77)'
else:
    farve = 'rgb(16, 174, 140)'

trace = go.Bar(
    x = [d + 10],

Could you please ask Neil then to look at it if it possible?

[olemunch]

@sashuk Could you please try to draw an example profile from the input below. When we draw this all colors are correct. It seems in the example above, that colors of layers are ok, but all colors on borehole and intake are wrong (random).

json_str = '{"coordinates": [[565977, 6372779], [567447, 6370030]], "DGU_nr": [" 6. 37A", " 6. 37B"," 6. 64B", " 6. 413", " 6. 64A", " 6. 414", " 6. 835", " 6. 412", " 6. 783", " 6. 751", " 6. 931", " 6. 954", " 6. 955"], "Profile_depth": -80, "layers":["Potentialekort_Aalborg_config.txt","FOHMJ_config.txt"],"overlap":[[[0,170], [250,3000]],[[120,290],[800,1000]]],"Compound":"1591"}'

[sashuk]

@olemunch Tried to use the provided string as an input for the profile script, the result:

[olemunch]

@sashuk You dont get any boreholeinformation - do you have access to the CALYPSO database?

I get this (rename to html) Layout CALYPSO profiles v5 kemi.txt

There is a pg connection string in the script, but its an axternal access that requires ip approval. Du you have access to run the script directly on the server?

[sashuk]

@olemunch I have the exact copy of profiletool_kemi_v5.py, and it calls the psycopg2.connect(dbname="jupiter", user="gc2", password="***********", host="watsonc.mapcentia.com") on line 397, but it seems like it does not have an access to boreholes indeed! Yes, I've ran the

python3.6 profiletool_kemi_v5.py '{"coordinates": [[565977, 6372779], [567447, 6370030]], "DGU_nr": [" 6. 37A", " 6. 37B"," 6. 64B", " 6. 413", " 6. 64A", " 6. 414", " 6. 835", " 6. 412", " 6. 783", " 6. 751", " 6. 931", " 6. 954", " 6. 955"], "Profile_depth": -80, "layers":["Potentialekort_Aalborg_config.txt","FOHMJ_config.txt"],"overlap":[[[0,170], [250,3000]],[[120,290],[800,1000]]],"Compound":"1591"}'

and it returned the exact same profile as before (without boreholes). So, somehow the script needs to have access to the database of boreholes?

However, if I run the profile as regular, it generates a profile with boreholes (and potential line)

[olemunch]

@sashuk What does it mean to run the tool as regular? It clearly does not use the correct input and draws a completely different profile?

@mapcentia will know how the program gets access to the database. I'm not sure how to get any further with this at the moment

[sashuk]

@olemunch The regular means not using the json_str as an input, but building the profile by myself using existing tools - the profile is created and has boreholes in it, so there is access to the boreholes data:

However, the script can not access data when it is called with the provided json_str.

[olemunch]

@sashuk Ok - so its exactly the same script? Perhaps its not relevant but if you send me the json input that was generated in your "regular" test I could try to run it from here.

[sashuk]

@olemunch Sure! The inputJSON in my case:

{"coordinates":[[547037.7528058332,6318983.168985754],[546807.2087636922,6316532.403667847],[545728.8328974872,6318855.314661024]],"DGU_nr":[" 34. 167"," 34. 2733"," 34. 398"," 34. 1223"," 34. 1224"," 34. 1225"," 34. 1410"," 34. 2324"," 34. 2683"," 34. 3591"," 34. 3679"," 34. 3680"," 34. 4090"," 34. 65"," 34. 768"],"Profile_depth":-100,"Compound":"1176","layers":["/usr/share/nginx/html/extensions/watsonc/scripts/intersection/../profil/Potentialekort_Aalborg_config.txt","/usr/share/nginx/html/extensions/watsonc/scripts/intersection/../profil/FOHMJ_config.txt"],"overlap":[[[0,5023]],[[0,5023]]]}

cleaned version without relative path:

{"coordinates":[[547037.7528058332,6318983.168985754],[546807.2087636922,6316532.403667847],[545728.8328974872,6318855.314661024]],"DGU_nr":[" 34. 167"," 34. 2733"," 34. 398"," 34. 1223"," 34. 1224"," 34. 1225"," 34. 1410"," 34. 2324"," 34. 2683"," 34. 3591"," 34. 3679"," 34. 3680"," 34. 4090"," 34. 65"," 34. 768"],"Profile_depth":-100,"Compound":"1176","layers":["Potentialekort_Aalborg_config.txt","FOHMJ_config.txt"],"overlap":[[[0,5023]],[[0,5023]]]}

Result:

[sashuk]

@olemunch Hello, Ole! I was experimenting and noticed a strange thing - when I use HEX color (not rgb(255,60,60)), coloring works correctly. Could you please specify if you have noticed something like this before?

[olemunch]

No i haven't tried that. Interessting :-)

according to https://plot.ly/javascript/line-charts/ the syntax for rgb coloring is: color: 'rgb(128, 0, 128)'

Is there something with the quotes ' ' that does not match the output from the script?

Otherwise we can change everything to HEX notification if that solves the problem

[sashuk]

@olemunch If following code is put in https://github.com/WatsonC-hub/profil/blob/master/profiletool_kemi_v5.py#L542 till 548 line:

if last_chem > data_limit['limit'][0]:
    farve ='#fc3c3c'
elif last_chem > data_limit['attention'][0]:
    farve = '#f7a84d'
else:
    farve = '#10ae8c'

Then the bar coloring works (quotes are kept the same).

I propose to replace color notation only in this place, as normally it should not depend on the color notation at all, so I am moving to show both maximum and the latest value in the bar (modifying the Python script to detect the latest value and display it as well).

[olemunch]

@sashuk I'm affraid the coloring of boreholes is a problem also. The rgb notation is used many places in the script. Lets get back to this in another issue if its still a problem - great if you move on.

[sashuk]

@olemunch Got it! The issue with coloring was created: https://github.com/sashuk/watsonc/issues/70

[olemunch]

@sashuk great!

[sashuk]

@mapcentia @olemunch @JacobBirk Implemented in https://github.com/WatsonC-hub/profil/commit/21a2287e51da77379636f386e45cbdb0dc6d1f22 (profil script was modified)

Demo: