error in panda.flash_release()

[nebukadnezar]

Need to reflash my panda white. Using panda.flash_release() does this:

Using panda with serial XXX
Fetching latest firmware from github.com/commaai/panda-artifacts
Fetching firmware from https://github.com/commaai/panda-artifacts/raw/master/panda-v1.7.3-DEV-d034f3e9-RELEASE.zip
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.7/site-packages/panda/flash_release.py", line 32, in flash_release
    path = io.StringIO(r.content)
TypeError: initial_value must be str or None, not bytes

[nebukadnezar]

I changed the string functions to byte functions and got it to halfway work - except it completely bricked my panda in the end.

Proceeded to install everything from scratch in a ubuntu 20.04 VM, and now am getting the following:

xxx@ubuntu:~$ sudo pip3 install --upgrade pandacan
[sudo] password for xxx: 
Requirement already up-to-date: pandacan in /usr/local/lib/python3.8/dist-packages/pandacan-0.0.9-py3.8.egg (0.0.9)
Requirement already satisfied, skipping upgrade: hexdump>=3.3 in /usr/local/lib/python3.8/dist-packages/hexdump-3.3-py3.8.egg (from pandacan) (3.3)
Requirement already satisfied, skipping upgrade: libusb1==1.6.6 in /usr/local/lib/python3.8/dist-packages/libusb1-1.6.6-py3.8.egg (from pandacan) (1.6.6)
Requirement already satisfied, skipping upgrade: pycryptodome>=3.9.8 in /usr/local/lib/python3.8/dist-packages/pycryptodome-3.9.8-py3.8-linux-x86_64.egg (from pandacan) (3.9.8)
Requirement already satisfied, skipping upgrade: requests in /usr/lib/python3/dist-packages (from pandacan) (2.22.0)
Requirement already satisfied, skipping upgrade: tqdm>=4.14.0 in /usr/local/lib/python3.8/dist-packages/tqdm-4.48.2-py3.8.egg (from pandacan) (4.48.2)
xxx@ubuntu:~$ PYTHONPATH="" sudo python3 -c "import panda; panda.flash_release()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/xxx/panda/python/flash_release.py", line 34, in flash_release
    zf = ZipFile(path)
  File "/usr/lib/python3.8/zipfile.py", line 1269, in __init__
    self._RealGetContents()
  File "/usr/lib/python3.8/zipfile.py", line 1332, in _RealGetContents
    endrec = _EndRecData(fp)
  File "/usr/lib/python3.8/zipfile.py", line 264, in _EndRecData
    fpin.seek(0, 2)
AttributeError: 'NoneType' object has no attribute 'seek' 

[robbederks]

Sorry for the long reaction time. I've fixed the issue with the flash_release script in commit 229047d. This is usually not the recommended way to flash your panda though (unless you want the latest release version, which is rarely updated. The main branch of panda is the latest stable code). The easiest way to do that is to call either make or make recover from your ./board folder.

[nebukadnezar]

That still doesn't work:

balt@ubuntu:~/panda$ git log -1
commit 229047d7c2938fd2c4cb2bf08ef43fb9b9b4b6dd (HEAD -> master, origin/master, origin/HEAD)
Author: Robbe Derks <robbe.derks@gmail.com>
Date:   Mon Sep 7 10:26:27 2020 +0200

    fix flash_release
balt@ubuntu:~/panda$ ./python/flash_release.py 
Fetching latest firmware from github.com/commaai/panda-artifacts
Fetching firmware from https://github.com/commaai/panda-artifacts/raw/master/panda-v1.7.3-DEV-d034f3e9-RELEASE.zip
File Name                                             Modified             Size
bootstub.panda.bin                             2020-02-05 11:47:44        13348
panda.bin                                      2020-02-05 11:47:44        34088
user1.bin                                      2020-02-05 11:47:46       248212
user2.bin                                      2020-02-05 11:47:46       248212
boot_v1.5.bin                                  2016-05-20 16:53:08         3232
version                                        2020-02-05 11:47:48           27
0. Preparing to flash v1.7.3-DEV-d034f3e9-RELEASE
1. Entering DFU mode
Traceback (most recent call last):
  File "./python/flash_release.py", line 74, in <module>
    flash_release(*sys.argv[1:])
  File "./python/flash_release.py", line 45, in flash_release
    panda = Panda(st_serial)
  File "/home/balt/.local/lib/python3.8/site-packages/panda/__init__.py", line 157, in __init__
    self.connect(claim)
  File "/home/balt/.local/lib/python3.8/site-packages/panda/__init__.py", line 203, in connect
    assert(self._handle is not None)
AssertionError
balt@ubuntu:~/panda$ lsusb
Bus 001 Device 002: ID 0e0f:000b VMware, Inc. VMware Virtual USB Video Device
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 009: ID bbaa:ddcc VMware, Inc. VMware Virtual USB Hub
Bus 002 Device 008: ID 0e0f:0008 VMware, Inc. VMware Virtual USB Mouse
Bus 002 Device 003: ID 0e0f:0002 VMware, Inc. Virtual USB Hub
Bus 002 Device 002: ID 0e0f:0003 VMware, Inc. Virtual Mouse
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
balt@ubuntu:~/panda$ 

and make recover fails as well:

balt@ubuntu:~/panda/board$ make recover
arm-none-eabi-gcc -g -Wall -Wextra -Wstrict-prototypes -Werror -mlittle-endian -mthumb -mcpu=cortex-m4 -mhard-float -DSTM32F4 -DSTM32F413xx -mfpu=fpv4-sp-d16 -fsingle-precision-constant -I inc -I ../ -nostdlib -fno-builtin -std=gnu11 -Os -Tstm32_flash.ld "-DALLOW_DEBUG" -o obj/startup_stm32f413xx.o -c startup_stm32f413xx.s
echo "const uint8_t gitversion[] = \"v1.7.5-DEV-229047d7-"DEBUG"\";" > obj/gitversion.h
../crypto/getcertheader.py ../certs/debug.pub ../certs/release.pub > obj/cert.h
arm-none-eabi-gcc -MT obj/bootstub.panda.o -MMD -MP -MF generated_dependencies/bootstub.Td -g -Wall -Wextra -Wstrict-prototypes -Werror -mlittle-endian -mthumb -mcpu=cortex-m4 -mhard-float -DSTM32F4 -DSTM32F413xx -mfpu=fpv4-sp-d16 -fsingle-precision-constant -I inc -I ../ -nostdlib -fno-builtin -std=gnu11 -Os -Tstm32_flash.ld "-DALLOW_DEBUG" -o obj/bootstub.panda.o -c bootstub.c
arm-none-eabi-gcc -g -Wall -Wextra -Wstrict-prototypes -Werror -mlittle-endian -mthumb -mcpu=cortex-m4 -mhard-float -DSTM32F4 -DSTM32F413xx -mfpu=fpv4-sp-d16 -fsingle-precision-constant -I inc -I ../ -nostdlib -fno-builtin -std=gnu11 -Os -Tstm32_flash.ld "-DALLOW_DEBUG" -o obj/sha.panda.o -c ../crypto/sha.c
arm-none-eabi-gcc -g -Wall -Wextra -Wstrict-prototypes -Werror -mlittle-endian -mthumb -mcpu=cortex-m4 -mhard-float -DSTM32F4 -DSTM32F413xx -mfpu=fpv4-sp-d16 -fsingle-precision-constant -I inc -I ../ -nostdlib -fno-builtin -std=gnu11 -Os -Tstm32_flash.ld "-DALLOW_DEBUG" -o obj/rsa.panda.o -c ../crypto/rsa.c
arm-none-eabi-gcc -g -Wall -Wextra -Wstrict-prototypes -Werror -mlittle-endian -mthumb -mcpu=cortex-m4 -mhard-float -DSTM32F4 -DSTM32F413xx -mfpu=fpv4-sp-d16 -fsingle-precision-constant -I inc -I ../ -nostdlib -fno-builtin -std=gnu11 -Os -Tstm32_flash.ld "-DALLOW_DEBUG" -o obj/bootstub.panda.elf obj/startup_stm32f413xx.o obj/bootstub.panda.o obj/sha.panda.o obj/rsa.panda.o
arm-none-eabi-objcopy -v -O binary obj/bootstub.panda.elf obj/bootstub.panda.bin
copy from `obj/bootstub.panda.elf' [elf32-littlearm] to `obj/bootstub.panda.bin' [binary]
arm-none-eabi-gcc -MT obj/main.panda.o -MMD -MP -MF generated_dependencies/main.Td -g -Wall -Wextra -Wstrict-prototypes -Werror -mlittle-endian -mthumb -mcpu=cortex-m4 -mhard-float -DSTM32F4 -DSTM32F413xx -mfpu=fpv4-sp-d16 -fsingle-precision-constant -I inc -I ../ -nostdlib -fno-builtin -std=gnu11 -Os -Tstm32_flash.ld "-DALLOW_DEBUG" -o obj/main.panda.o -c main.c
arm-none-eabi-gcc -Wl,--section-start,.isr_vector=0x8004000 -g -Wall -Wextra -Wstrict-prototypes -Werror -mlittle-endian -mthumb -mcpu=cortex-m4 -mhard-float -DSTM32F4 -DSTM32F413xx -mfpu=fpv4-sp-d16 -fsingle-precision-constant -I inc -I ../ -nostdlib -fno-builtin -std=gnu11 -Os -Tstm32_flash.ld "-DALLOW_DEBUG" -o obj/panda.elf obj/startup_stm32f413xx.o obj/main.panda.o
arm-none-eabi-objcopy -v -O binary obj/panda.elf obj/code.bin
copy from `obj/panda.elf' [elf32-littlearm] to `obj/code.bin' [binary]
SETLEN=1 ../crypto/sign.py obj/code.bin obj/panda.bin ../certs/debug
signing 43484 bytes
hash: 9b0d13c5e25dfe0611721ce8323a46f1589fe50a
PYTHONPATH=../ python3 -c "from python import Panda; Panda().reset(enter_bootstub=True); Panda().reset(enter_bootloader=True)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/balt/panda/python/__init__.py", line 157, in __init__
    self.connect(claim)
  File "/home/balt/panda/python/__init__.py", line 203, in connect
    assert(self._handle is not None)
AssertionError
make: [build.mk:46: recover] Error 1 (ignored)
sleep 1.0
"dfu-util" -d 0483:df11 -a 0 -s 0x08004000 -D obj/panda.bin
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
dfu-util: No DFU capable USB device available
make: *** [build.mk:48: recover] Error 74

[nebukadnezar]

I'm getting no love. I've now reinstalled an Ubuntu 16 VM from scratch, cloned the git repo, installed pip3 and installed pandacan from the git with sudo pip3 install ./ and am getting a new set of errors - both when trying to run flash_release and make / make recover:

balt@ubuntu:~/panda$ python3 ./python/flash_release.py 
Traceback (most recent call last):
  File "./python/flash_release.py", line 74, in <module>
    flash_release(*sys.argv[1:])
  File "./python/flash_release.py", line 10, in flash_release
    from panda import Panda, PandaDFU
  File "/usr/local/lib/python3.5/dist-packages/panda/__init__.py", line 45
    print(f"  R 0x{address:x}: 0x{dddat.hex()}")
                                              ^
SyntaxError: invalid syntax

I repeat my plea from 2 weeks ago: Can someone please post step by step instructions to get this firmware built and flashed?

[robbederks]

From this log and the one from your other issue, it seems that your panda is not showing up in DFU mode (no devices in lsusb). Are you connecting with a panda paw?

[nebukadnezar]

yes, connecting with a paw.

[robbederks]

Oh, I see you are using a VM to run ubuntu? When the panda goes into DFU mode, it re-enumerates over USB as another device. Are you sure that device is passed on to your VM?

[nebukadnezar]

Yes I pass all USB devices, however, that's no longer an issue. I've dusted off an old physical hardware Ubuntu 16 laptop.

I installed panda from git on there by first cloning the repo then

cd panda; pip3 install ./

which seemed to install panda plus dependencies, no errors reported.

However, when I run python3 and try to import panda:

~/panda$ python3
Python 3.5.2 (default, Oct  8 2019, 13:06:37) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import panda
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/balt/.local/lib/python3.5/site-packages/panda/__init__.py", line 45
    print(f"  R 0x{address:x}: 0x{dddat.hex()}")
                                              ^
SyntaxError: invalid syntax
>>> 

[robbederks]

I see. The issue you're having now is the version of python being too old. We rely on having 3.7 or newer.

[nebukadnezar]

Ok, so I've reinstalled this laptop to Ubuntu 20.04 LTS.

balt@phoenix:~$ python3 -V
Python 3.8.2

installed pip3: sudo apt install python3-pip cloned panda: git clone https://github.com/commaai/panda.git

installed panda using pip3 from the cloned directory:

balt@phoenix:~/panda$ pip3 install ./
Processing /home/balt/panda
Collecting hexdump>=3.3
  Using cached hexdump-3.3.zip (12 kB)
Collecting libusb1==1.6.6
  Using cached libusb1-1.6.6.tar.gz (56 kB)
Collecting pycryptodome>=3.9.8
  Downloading pycryptodome-3.9.8-cp38-cp38-manylinux1_x86_64.whl (13.7 MB)
     |████████████████████████████████| 13.7 MB 4.8 MB/s 
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from pandacan==0.0.9) (2.22.0)
Collecting tqdm>=4.14.0
  Downloading tqdm-4.50.0-py2.py3-none-any.whl (70 kB)
     |████████████████████████████████| 70 kB 1.5 MB/s 
Building wheels for collected packages: pandacan, hexdump, libusb1
  Building wheel for pandacan (setup.py) ... done
  Created wheel for pandacan: filename=pandacan-0.0.9-py2.py3-none-any.whl size=21020 sha256=08423e7bf208ad87deb06cf2bae61d84f762eeff926613a89c2f2c5f0163d157
  Stored in directory: /tmp/pip-ephem-wheel-cache-7asakmi5/wheels/c1/ee/28/7ed8fdac6ae14e73a3a4ca427aea7a6a644190fd75ee46c149
  Building wheel for hexdump (setup.py) ... done
  Created wheel for hexdump: filename=hexdump-3.3-py3-none-any.whl size=8911 sha256=1d389d76d40b96bab71b4cb271b974932b56e3230ae4fdbc066c07f77a605b07
  Stored in directory: /home/balt/.cache/pip/wheels/7f/b7/72/1cc327e831ffef71bca1b0ca5e40b68471875f740ec9270c0f
  Building wheel for libusb1 (setup.py) ... done
  Created wheel for libusb1: filename=libusb1-1.6.6-py3-none-any.whl size=58455 sha256=2dec426aecd09520483030abff5cf525723994a19595da9db4f7fba2a833d9d6
  Stored in directory: /home/balt/.cache/pip/wheels/7d/a3/06/06165ef360de4bf75a47eae768f9e640557ac20b7cae2a1be9
Successfully built pandacan hexdump libusb1
Installing collected packages: hexdump, libusb1, pycryptodome, tqdm, pandacan
  WARNING: The script tqdm is installed in '/home/balt/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed hexdump-3.3 libusb1-1.6.6 pandacan-0.0.9 pycryptodome-3.9.8 tqdm-4.50.0

plugged in my panda+paw:

balt@phoenix:~/panda$ lsusb
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp. 4-Port USB 3.0 Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 005: ID 04f2:b580 Chicony Electronics Co., Ltd HD WebCam
Bus 001 Device 004: ID 2386:3115 Raydium Corporation Raydium Touch System
Bus 001 Device 003: ID 0489:e09f Foxconn / Hon Hai 
Bus 001 Device 006: ID bbaa:ddcc  
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp. 4-Port USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

and trying to flash it to the release:

balt@phoenix:~/panda$ python3
Python 3.8.2 (default, Jul 16 2020, 14:00:26) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import panda
>>> panda.flash_release()
Fetching latest firmware from github.com/commaai/panda-artifacts
Fetching firmware from https://github.com/commaai/panda-artifacts/raw/master/panda-v1.7.3-DEV-d034f3e9-RELEASE.zip
File Name                                             Modified             Size
bootstub.panda.bin                             2020-02-05 11:47:44        13348
panda.bin                                      2020-02-05 11:47:44        34088
user1.bin                                      2020-02-05 11:47:46       248212
user2.bin                                      2020-02-05 11:47:46       248212
boot_v1.5.bin                                  2016-05-20 16:53:08         3232
version                                        2020-02-05 11:47:48           27
0. Preparing to flash v1.7.3-DEV-d034f3e9-RELEASE
1. Entering DFU mode
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/balt/.local/lib/python3.8/site-packages/panda/flash_release.py", line 45, in flash_release
    panda = Panda(st_serial)
  File "/home/balt/.local/lib/python3.8/site-packages/panda/__init__.py", line 157, in __init__
    self.connect(claim)
  File "/home/balt/.local/lib/python3.8/site-packages/panda/__init__.py", line 203, in connect
    assert(self._handle is not None)
AssertionError
>>> 

Any help getting this to work is greatly appreciated!

[nebukadnezar]

Looks like a sudo was missing and now it works. HOWEVER: The flashed version now only offers a BROKEN wifi identifier and the while connecting to the pairing wifi works, the subsequent attempt to connect to the BROKEN wifi does not. The password appears to be different.

So the question now is how to flash a non-broken version!?

balt@phoenix:~/panda$ sudo -E python3 -c "import panda; panda.flash_release()"
Fetching latest firmware from github.com/commaai/panda-artifacts
Fetching firmware from https://github.com/commaai/panda-artifacts/raw/master/panda-v1.7.3-DEV-d034f3e9-RELEASE.zip
File Name                                             Modified             Size
bootstub.panda.bin                             2020-02-05 11:47:44        13348
panda.bin                                      2020-02-05 11:47:44        34088
user1.bin                                      2020-02-05 11:47:46       248212
user2.bin                                      2020-02-05 11:47:46       248212
boot_v1.5.bin                                  2016-05-20 16:53:08         3232
version                                        2020-02-05 11:47:48           27
0. Preparing to flash v1.7.3-DEV-d034f3e9-RELEASE
1. Entering DFU mode
opening device 340059000e51363338383037 0xddcc
connected
opening device 340059000e51363338383037 0xddee
connected
2. Programming bootstub
programming 0 with length 2048
programming 1 with length 2048
programming 2 with length 2048
programming 3 with length 2048
programming 4 with length 2048
programming 5 with length 2048
programming 6 with length 2048
3. Flashing main code
opening device 340059000e51363338383037 0xddee
connected
flash: main version is v1.7.3-DEV-d034f3e9-RELEASE
flash: bootstub version is v1.7.3-DEV-d034f3e9-RELEASE
flash: unlocking
flash: erasing
flash: flashing
flash: resetting
opening device 340059000e51363338383037 0xddcc
connected
4. Verifying version
opening device 340059000e51363338383037 0xddcc
connected
dongle id: 3d20691fdb0857ff
v1.7.3-DEV-d034f3e9-RELEASE should be v1.7.3-DEV-d034f3e9-RELEASE
6. Success!

[robbederks]

Hi, the reason you had to run this with sudo is because you presumably don't have udev rules for the DFU usb device which allow access for non-root users. Wifi has been deprecated on the white pandas, you can flash an older version if you'd want to use this: #592

[nebukadnezar]

ok, but the version I flashed is 1.7.3-DEV which is 8 months old - long before the wifi support was dropped on Aug 15?

[nebukadnezar]

Ok, that seems to have worked. Download an old image from https://github.com/commaai/panda-artifacts and then run this pointing to the firmware zip file, but running the current (i.e. fixed) branch:

sudo -E python3 -c "import panda; panda.flash_release('/home/balt/Downloads/panda-v1.5.2-DEV-736c2cbf-RELEASE.zip')"

and yes for some reason I still need to sudo this, even though I've installed the udev rules.

Why are you discontinuing wifi support? That's a pretty major feature deprecation making the panda unusable with logger software on iPhone.

[Jahnkeanater]

Finally someone is figuring out this issue! My panda has been BROKEN for 6 months ever since I tried to update using curl.

[adeebshihadeh]

We have no use for wifi anymore, so it made sense to deprecate it. It's difficult to maintain something that you don't use. The old firmware releases and all the code is still available.

[nebukadnezar]

If anyone is looking for CAN bus decoder with wifi and on board logging have a look here http://www.jwardell.com/canserver/ It doesn't write to the CAN bus (a safety/security plus for some of us) but evidently wouldn't work for the comma.ai environment.