Pip upgrades and served image size reductions

[JeffAbrahamson]

Hmm, there are two issues:

• a couple photo tests are failing for somewhat obscure reasons.
• the github lint runner is unhappy with the pip version (I think)

[JeffAbrahamson]

@Shriukan33 On the off chance you have time and motivation to look at this, there's an old test you wrote that appears to be unrelated to my change and that's failing by not finding the button with id="upvote-button". Though the id certainly seems to be there!

python manage.py migrate && TEST_MODE=1 python manage.py test photo.tests

======================================================================
ERROR: test_vote_anon (photo.tests.TestVotes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/vagrant/transport_nantes/photo/tests.py", line 486, in test_vote_anon
    self.anon_browser.find_element(By.ID, "upvote-button").click()
  File "/vagrant/venv.vagrant/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 1244, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "/vagrant/venv.vagrant/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "/vagrant/venv.vagrant/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="upvote-button"]"}

[MourguesBenjamin]

• the github lint runner is unhappy with the pip version (I think)

The pip message is at bottom but if you read carefully it's the building of anyjson that is problematic error in anyjson setup command: use_2to3 is invalid.

This is because 2to3 has been removed from setuptools>58, and you need to downgrade it

[MourguesBenjamin]

About the unrelated test, I remember Selenium being subject to concurrency despite my best efforts to fix it. Sometimes this will fail, sometime the page will load. It's possible that if you remove the headless option locally the test passes green again.

It seems to be one of those time things aren't going exactly right, but I wouldn't know how to fix it, I tried many solutions from selenium, like action chains, waits and so on, nothing worked out consistently.

[JeffAbrahamson]

Thanks for the tips. I love race conditions. ;-)

What I'm currently seeing is that the test harness creates a PhotoEntry object, saves it, PhotoEntry.objects.all() shows that it's there, but when a separate bit of code modifies the object based on a separate query, the saved object isn't visible to the test.

Outside of test mode, the saves are definitely working, I can see them in the database.

I remember something about selenium having its own special relationship with databases...

[MourguesBenjamin]

I don't think it was related to the database, more like the element upvote-button for some mysterious reason isn't seen by Selenium while it wants to click on it to upvote. Selenium doesn't find the button (but why?) and test fails.

Edit : I remember something to do with db locks indeed, it definitely pain to work with

[JeffAbrahamson]

• the github lint runner is unhappy with the pip version (I think)

The pip message is at bottom but if you read carefully it's the building of anyjson that is problematic error in anyjson setup command: use_2to3 is invalid.

This is because 2to3 has been removed from setuptools>58, and you need to downgrade it

This is also an annoyingly strange one, as setuptools isn't in requirements.txt, does show up in pip list but doesn't in pip freeze. I didn't realise that last (difference between list and freeze) was possible.

[JeffAbrahamson]

Hmm, @Shriukan33 , do you remember why you sometimes say self.client (e.g., L275) and sometimes self.auth_client (e.g., L279)?

They appear to be different clients that don't know about each other's objects.

[MourguesBenjamin]

Hmm, @Shriukan33 , do you remember why you sometimes say self.client (e.g., L275) and sometimes self.auth_client (e.g., L279)?

They appear to be different clients that don't know about each other's objects.

This is because they are indeed different ! self.client is a default prop that is an anonymous session, while auth client is a client that logged in / connected to a user account

See it as two browser sessions

[JeffAbrahamson]

Hmm, @Shriukan33 , do you remember why you sometimes say self.client (e.g., L275) and sometimes self.auth_client (e.g., L279)? They appear to be different clients that don't know about each other's objects.

This is because they are indeed different ! self.client is a default prop that is an anonymous session, while auth client is a client that logged in / connected to a user account

I think the bug I've hit (in the tests) is that I modify the db in a view, but only one client knows about it.

The actual test I want has two clients accessing the same database. But I seem to have two clients accessing two databases.

[MourguesBenjamin]

Between two tests, the database is reset according to the setUp() method

If you modify any data in a test, it's cancelled at the end of the test.

[JeffAbrahamson]

Between two tests, the database is reset according to the setUp() method

If you modify any data in a test, it's cancelled at the end of the test.

Yup, but look at this test:

Note that it alternates between auth and non-auth clients.

[MourguesBenjamin]

Self.client is defined in the base class, and we add an auth client on top of it While in the same test, the two affect and query the same database new test = new db

[JeffAbrahamson]

Aha, got it.

Once the object is saved, it's not the same object as what is fetched. In the original code, this didn't matter, but it wasn't robust. When I added some code that scanned the db and made some changes, the test broke.

[JeffAbrahamson]

So, basically, this was several hours of debugging to eventually write self.photo_entry.refresh_from_db(). 😅

[JeffAbrahamson]

• the github lint runner is unhappy with the pip version (I think)

The pip message is at bottom but if you read carefully it's the building of anyjson that is problematic error in anyjson setup command: use_2to3 is invalid. This is because 2to3 has been removed from setuptools>58, and you need to downgrade it

This is also an annoyingly strange one, as setuptools isn't in requirements.txt, does show up in pip list but doesn't in pip freeze. I didn't realise that last (difference between list and freeze) was possible.

Any clue on this? This is quite strange.

requirements.txt requires neither 2to3 (it does require six) nor setuptools. Nor anyjson, which is a dependency from elsewhere.

I added a line setuptools<58.0.0, but that doesn't change the anyjson error.

<scratching head>