Dafang (hacked) camera still cgi image issue

[TheZoker]

The problem

Since update to the latest beta 2022.04b0 the generic camera is not available as yaml config anymore. So I tried to use the new integration. The still image url for my dafang hacked camera looks like this: https://<ip-adress>/cgi-bin/currentpic.cgi

This worked very well in HA in the past. But since the change, HA tells me, that the still image is not valid and I can't add it to the integration config.

What version of Home Assistant Core has the issue?

core-2022.4.0b0

What was the last working version of Home Assistant Core?

core-2022.3.8

What type of installation are you running?

Home Assistant OS

Integration causing the issue

generic

Link to integration documentation on our website

https://www.home-assistant.io/integrations/generic

Diagnostics information

No response

Example YAML snippet

platform: generic
name: Kamera
username: user
password: pass
authentication: basic
still_image_url: https://<ip-adress>/cgi-bin/currentpic.cgi
stream_source: rtsp://<ip-adress>:8554/unicast
verify_ssl: false
scan_interval: 5

Anything in the logs that might be useful for us?

Logger: libav.mp3float
Source: /usr/local/lib/python3.9/concurrent/futures/thread.py:58
First occurred: 00:01:33 (1 occurrences)
Last logged: 00:01:33

Header missing

--- 

Logger: async_upnp_client.client
Source: /usr/local/lib/python3.9/site-packages/async_upnp_client/client.py:419
First occurred: 00:01:34 (1 occurrences)
Last logged: 00:01:34

Got invalid value for <UpnpStateVariable(PlaybackStorageMedium, string)>: NETWORK,NONE

---

Logger: homeassistant.components.generic.config_flow
Source: components/generic/config_flow.py:292
Integration: Generic Camera (documentation, issues)
First occurred: 00:01:36 (1 occurrences)
Last logged: 00:01:36

Error importing generic IP camera platform config: unexpected error '['invalid_still_image']'

---

Logger: aiohttp.server
Source: components/stream/__init__.py:86
First occurred: 00:04:36 (1 occurrences)
Last logged: 00:04:36

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 222, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 219, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 137, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 724, in get
    return await self.handle(request, camera)
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 764, in handle
    stream = await camera.handle_async_mjpeg_stream(request)
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 589, in handle_async_mjpeg_stream
    return await self.handle_async_still_stream(request, self.frame_interval)
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 577, in handle_async_still_stream
    return await async_get_still_stream(
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 252, in async_get_still_stream
    img_bytes = await image_cb()
  File "/usr/src/homeassistant/homeassistant/components/generic/camera.py", line 176, in async_camera_image
    await self.async_create_stream()
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 526, in async_create_stream
    self.stream = create_stream(
  File "/usr/src/homeassistant/homeassistant/components/stream/__init__.py", line 86, in create_stream
    raise HomeAssistantError("Stream integration is not set up.")
homeassistant.exceptions.HomeAssistantError: Stream integration is not set up.

Additional information

No response

[probot-home-assistant[bot]]

generic documentation generic source _{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @davet2001, mind taking a look at this issue as it has been labeled with an integration (generic) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[davet2001]

@TheZoker thanks for raising the issue! Please can you confirm what type of still image it is?

If possible could you view it in a browser and save a copy to the ticket?

Many thanks!

[TheZoker]

Thanks for the quick answer :)

Sure thing, here is the image:

I'm not sure if GitHub changes anything about the image during upload

[bdraco]

It would be good to zip it up before attaching to make sure GitHub didn't modify it

[davet2001]

Looks like there is a bug in the underlying imghdr library meaning it can't recognise some jpeg types.

https://github.com/lektor/lektor/issues/624 also some info here: https://stackoverflow.com/questions/36870661/imghdr-python-cant-detec-type-of-some-images-image-extension

I tried out this code:

def get_image_type(image):
    """Get the format of downloaded bytes that could be an image."""
    fmt = imghdr.what(None, h=image)
    if fmt is None:
        # if imghdr can't figure it out, could be svg.
        with contextlib.suppress(UnicodeDecodeError):
            if image.decode("utf-8").lstrip().startswith("<svg"):
                return "svg+xml"
        # JPEG data in JFIF format
        if b'JFIF' in image[:23]: 
            return 'jpeg1'
        JPEG_MARK = b'\xff\xd8\xff\xdb\x00C\x00\x08\x06\x06' \
        b'\x07\x06\x05\x08\x07\x07\x07\t\t\x08\n\x0c\x14\r\x0c\x0b\x0b\x0c\x19\x12\x13\x0f'
        # JPEG with small header
        if len(image) >= 32 and 67 == image[5] and image[:32] == JPEG_MARK:
            return 'jpeg2'
        # JPEG data in JFIF or Exif format"""
        if image[6:10] in (b'JFIF', b'Exif'):
            return 'jpeg3'
        if image[:2] == b'\xff\xd8':
            return 'jpeg4'
    return fmt

And on your sample image it returned jpeg4, which means imghdr.what() could not work out the image format, but it started with two characters that are a valid jpeg.

Will have another look again this evening.

[mib1185]

imghdr is deprecated as per PEP 594, so there won't be further enhancements to it.

source: https://bugs.python.org/issue28591#msg415044

---

as per https://peps.python.org/pep-0594/#deprecated-modules following replacements are suggested:

• filetype
• puremagic
• python-magic

[davet2001]

@mib1185 Thanks for the suggestion. Yes, it seems imghdr is not a reliable library.

I have also been testing using PIL which seems pretty reliable:

    fmt = imghdr.what(None, h=image)
    if fmt is None:
        imagefile = io.BytesIO(image)
        with contextlib.suppress(PIL.UnidentifiedImageError):
            img = PIL.Image.open(imagefile)
            fmt = img.format.lower()

    if fmt is None:
        # if imghdr can't figure it out, could be svg.
        with contextlib.suppress(UnicodeDecodeError):
            if image.decode("utf-8").lstrip().startswith("<svg"):
                return "svg+xml"

The above also works but will do a bit more research before settling on a preferred option.

[TheZoker]

If you have a custom component I can test, let me know :)

[balloob]

Duplicate of #69011

[davet2001]

If you have a custom component I can test, let me know :)

@TheZoker If you can test the branch that would be perfect! Se above link for details.