Canvas together with session_state

[chraibi]

I'm trying to figure out how to use a "stateful"-canvas.

So far that works for me

        if st.session_state.bg_img is None:
            logging.info("START new canvas")
            bg_img = create_bg_img()
            st.session_state.bg_img = bg_img
        else:
            bg_img = st.session_state.bg_img

        canvas = st_canvas(
            fill_color="rgba(255, 165, 0, 0.3)",
            stroke_width=stroke_width,
            stroke_color="#060EE8",
            background_color="#eee",
            background_image=bg_img,
            update_streamlit=True,
            width=img_width,
            height=img_height,
            drawing_mode=drawing_mode,
            key="canvas",
        )

This works well, but still I would like to not create the canvas again and again. Saving the canvas itself as a session_State-variable makes changes its type to dict. Strange!

Maybe there is somewhere an example, but I did not find it. This was was removed: :-) https://github.com/andfanilo/streamlit-drawable-canvas-demo/blob/master/SessionState.py

[andfanilo]

Hi @chraibi

I'm not totally sure what you tried to do with the background image 😃 but if I understand your stateful problem correctly, you can put canvas_result.json_data in session state and when you reset your canvas (by changing the key variable for example) reimport the state wih initial_drawing (like in https://github.com/andfanilo/streamlit-drawable-canvas-demo/blob/6b5f6371375b7dc404d4510f094359dd68bdab8c/app.py#L122)

I have not tested as I'm in seminar for the week but that may work! Fanilo

[chraibi]

Thanks for your answer, Fanilo. Basically what I'm doing, is load some trajectory-files, plot them in a canvas, and then draw on the canvas. Switching between different trajectory files, yield background images that are distorted. Therefore, I was thinking of ways to update the canvas as an object, not only it's data.

Here, an example.

First I load this file:

Then this one (note the dimensions are not right)

The geometry looks like this one:

I'm not sure if I could explain my issue clearly, if not, you can have a try by yourself https://share.streamlit.io/pedestriandynamics/dashboard/main/app.py

Enjoy your seminar! :-)

[DirkRemmers]

Hello @andfanilo and @chraibi,

I think I have another example that might help to clarify what @chraibi means with a 'stateful' canvas.

In my use, I'm using st_canvas to do a very simple task: put lines over interesting regions of an image, that can later be used to analyze the intensities of these line segments. The goal is to do this on multiple images in one session, that I can 'label' after each other.

Let's say we want to label 2 images, I would use something similar to the following code:

# select the image
image_list = [image1, image2]
image_nr = st.number_input('select the image nr', min_value = 0, max_value = int(len(image_list)-1), step = 1)
image = image_list[image_nr]

# make a canvas for the selected image
update_to_streamlit = st.button('Update canvas to streamlit')
canvas_result = st_canvas(
     stroke_width=3,
     stroke_color="#05BEF5",
     background_image=image,
     update_streamlit=update_data,
     height=image.size[0],
     width=image.size[1],
     drawing_mode='line',
     key= f"{image_nr}_canvas"
      )

# save the line data for future processing
if 'results' not in st.session_state.keys():
     st.session_state['results'] = {}
if canvas_result.json_data is not None:
     st.session_state['results][f"results for image {image_nr}"] = canvas_result.json_data['objects']

This would work nicely when labeling one image completely before moving on to the next one (and so on). However, when I would like to go back and adapt one of my previously labeled images, things get a bit difficult.

Lets say I am already busy labeling image 2, and suddenly I realize that I forgot to label one important part of image 1. When I now go back to image 1, all the previous data I created will be gone. Somehow, st_canvas does not remember the lines I have previously drawn. It seems like the unique key parameter is not doing anything in this case.

Luckily there already is some functionality which would be able to help us here according to the documentation. This is where the initial_drawing parameter could come in handy. However, I'm having some issues with the implementation of this particular parameter. What I had in mind, was using the combination of the unique key and the initial_drawing parameters to create a canvas that could remember previous states, but my approach does not seem to work.

My attempt:

# select the image
image_list = [image1, image2]
image_nr = st.number_input('select the image nr', min_value = 0, max_value = int(len(image_list)-1), step = 1)
image = image_list[image_nr]

# set the default lines to None, but this value will be replaced by the .json_data after this has been created
if f"{image_nr}_canvas_lines" not in st.session_state.keys():
     st.session_state[f"{image_nr}_canvas_lines"] = None

# make a canvas for the selected image
update_to_streamlit = st.button('Update canvas to streamlit')
canvas_result = st_canvas(
     stroke_width=3,
     stroke_color="#05BEF5",
     background_image=image,
     update_streamlit=update_data,
     height=image.size[0],
     width=image.size[1],
     drawing_mode='line',
     initial_drawing=st.session_state[f"{image_nr}_canvas_lines"],
     key= f"{image_nr}_canvas"
      )

# save the line data for future processing
if 'results' not in st.session_state.keys():
     st.session_state['results'] = {}
if canvas_result.json_data is not None:
     st.session_state[f"{image_nr}_canvas_lines"] = canvas_result.json_data 
     # or one could (in theory) use st.session_state[f"{image_nr}_canvas_lines"] = st.session_state[f"{image_nr}_canvas"] 
     # but this does not work either
     st.session_state['results][f"results for image {image_nr}"] = canvas_result.json_data['objects']

I think it would be nice if the state of the canvas could be saved in the key by default (also the initial_drawing parameter). I don't really understand why it would make sense to reset the key, and use the initial_drawing parameter that is saved in the st.session_state instead (or is this not what @andfanilo meant in the first reply to this issue?) It might be important to tackle the interesting thing that happens when you try to put the canvas_result from my previous examples in st.session_state here. As @chraibi mentioned already, the type changes to dict.

Another thing that happens when I run the second code example, is that the app keeps re-running. Could you describe why this happens? (I don't think this is issue related though)

Nevertheless, I really like this streamlit component! Thank you very much for taking the time and putting in the effort to make this amazing streamlit addition!

Cheers, Dirk