Integrate with esp8266/Arduino

[probonopd]

Great project - I hope it can be nicely integrated with https://github.com/esp8266/Arduino (e.g., to do the uploading and "serial" monitor using just the ESP and no serial adapter).

I also cross-posted this at https://github.com/esp8266/Arduino/issues/2375

[probonopd]

/me would like to see a YT live stream with @cnlohr @igrr and @me-no-dev :-)

[me-no-dev]

@probonopd if I am able to do what I wrote above, neither the Host side, nor the ESP side would be a problem for bootloader uploading. Maybe even exactly emulate Arduino (with resetting to bootloader and uart) as I have done it before on AVR and STM32

[cnlohr]

@me-no-dev Unless you know how to get CDC to run over low-speed USB, it's going to be difficult. Do you? Because that would totes be rad!

Callbacks aren't quite what you'd expect, but they are close. This has to do with needing to reply on the USB bus IMMEDIATELY. It is similar to V-USB that way... The facility is sort of there, though. For example...

In bootloader/main.c, line 34... it does handle control messages kind of like what you're after.

void usb_handle_custom_control( int bmRequestType, int bRequest, int wLength, struct usb_internal_state_struct * ist )
{
    struct usb_urb * s = (struct usb_urb *)ist->usb_buffer;
    struct usb_endpoint * e = ist->ce;
    struct USBControlStruct * cc = &cctrl;

    if( bmRequestType == 0x80 )
    {
        if( bRequest == 0xa0) //US TO HOST "in"
        {
            if( cc->length_ret )
            {
                e->ptr_in = cc->ret;
                e->size_in = cc->length_ret;
                e->transfer_in_done_ptr == &cc->ret_done;
                if( wLength < e->size_in ) e->size_in = wLength;
                cc->length_ret = 0;
            }
        }
    }

    if( bmRequestType == 0x00 )
    {
        if( bRequest >= 0xa0 && bRequest < 0xc0 && cc->length_acc == 0 ) //HOST TO US "out" Only permit if we've already cleared out the message.
        {
            cc->acc_request = bRequest;
            cc->acc_value = s->wValue;
            cc->acc_index = s->wIndex;
            e->ptr_out = cc->acc;
            e->max_size_out = CUSTOM_BUFFERSIZE;
            if( e->max_size_out > wLength ) e->max_size_out = wLength;
            e->got_size_out = 0;
            e->transfer_done_ptr = &cc->length_acc;
        }
        else
        {
            e->ptr_out = 0;
            e->max_size_out = 0;
        }
    }
}

BUT!!! Keep in mind, this code has to execute /very/ fast! Code a little further on in main() is what actually handles the transaction.

        if( cc->length_acc && cc->ret_done ) //XXX Todo: Make sure our response was received.
        {
            switch( cc->acc_request )
            {
            case 0xa0: //Echo
                ets_memcpy( cc->ret, cc->acc, cc->length_acc );
                cc->ret = usb_custom_ret;
                cc->length_ret = cc->length_acc;
                cc->length_acc = 0; //Clear out the incoming data once done processing.
                printf( "/%d MSG:%s %d %02x:%04x:%04x\n", cc->length_acc, cc->acc, usb_internal_state.there_is_a_host,cc->acc_request,cc->acc_value,cc->acc_index );
                break;
            case 0xa1: //Read RAM.
                cc->ret = (uint8_t*)((cc->acc_value<<16) | cc->acc_index);
                cc->length_ret = cc->length_acc;
                cc->length_acc = 0;
                break;
            case 0xa2: //Read FLASH
            {
                int toread = (cc->acc_value>>8)*16;
                int addy = (((cc->acc_value & 0x0f)<<16) | cc->acc_index)*4;
                cc->ret = usb_custom_ret;
                SPIRead( addy, (uint32_t*)cc->ret, toread );
                cc->length_ret = toread;
                cc->length_acc = 0;
                break;

I won't be free until Wednesday, late or maybe Thursday :-/

[SpiraMirabilis]

Most OSes will be fine with a low speed USB CDC device, even though it technically breaks standard since bulk endpoints are forbidden. I'd be more worried that you can't guarantee any real time compliance since the esp steals cycles for its wifi functions.

[me-no-dev]

bulk endpoints are not forbidden, but are rather limited in amount of data and frequency of polling, which in reality makes only bauds below 57600 or so work (do not remember what I had calculated). Bulk endpoints are used in many/most low speed usb devices :) Please correct me if I'm wrong

[cnlohr]

It would not enumerate my serial CDC device at all. Something about "insufficient bandwidth" even when I wasn't even using the bulk endpoint. Anyone know if there's any "trick" around this?

EDIT Anyway, sounds like control messages make much more efficient use of the bandwidth!

[me-no-dev]

@cnlohr what OS did you try that on? BTW did you get my email?

[cnlohr]

@me-no-dev - Yes, I got it. Things have been insane recently, and until MAGFest is over I don't think I'll get a chance to try it. I was running on either linux 3.x or 4.4. I did notice your descriptors were different than mine. When you tested it, did it work in Windows. Is it okay for you to post the zip here so @leopck can give it a crack?

[leopck]

Would be great if I could get more leads to this, since I've been stuck at this issue for a while.

EDIT: I got it to enumerate, but it shows these errors.

I've been getting this issue and I can't go over it. Any ideas?

[ 1331.845052] ------------[ cut here ]------------
[ 1331.845068] WARNING: CPU: 7 PID: 862 at /build/linux-a2WvEb/linux-4.4.0/drivers/usb/core/urb.c:449 usb_submit_urb.part.6+0x142/0x560()
[ 1331.845072] usb 2-1.2: BOGUS urb xfer, pipe 3 != type 1
[ 1331.845075] Modules linked in: cdc_acm cp210x usbserial rfcomm drbg ansi_cprng ctr ccm bbswitch(OE) bnep nvidia_uvm(POE) nvidia_modeset(POE) nvidia(POE) arc4 snd_hda_codec_hdmi snd_hda_codec_realtek ath9k snd_hda_codec_generic intel_rapl ath9k_common x86_pkg_temp_thermal uvcvideo ath9k_hw snd_hda_intel snd_usb_audio intel_powerclamp videobuf2_vmalloc coretemp videobuf2_memops snd_usbmidi_lib kvm_intel snd_hda_codec ath videobuf2_v4l2 snd_hda_core mac80211 videobuf2_core snd_hwdep btusb kvm v4l2_common snd_pcm btrtl snd_seq_midi videodev btbcm snd_seq_midi_event btintel snd_rawmidi media bluetooth cfg80211 snd_seq irqbypass crct10dif_pclmul snd_seq_device crc32_pclmul snd_timer jmb38x_ms memstick mei_me snd joydev input_leds mei soundcore shpchp lpc_ich serio_raw cryptd binfmt_misc ideapad_laptop
[ 1331.845172]  sparse_keymap mxm_wmi wmi mac_hid parport_pc ppdev lp parport autofs4 hid_generic usbhid hid i915 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops psmouse sdhci_pci ahci drm sdhci tg3 libahci ptp pps_core video fjes
[ 1331.845215] CPU: 7 PID: 862 Comm: ModemManager Tainted: P           OE   4.4.0-36-generic #55-Ubuntu
[ 1331.845220] Hardware name: LENOVO 20090                           /Base Board Product Name, BIOS 47CN28WW(V2.06) 06/14/2011
[ 1331.845224]  0000000000000286 00000000aa160979 ffff8802097fba08 ffffffff813f13b3
[ 1331.845231]  ffff8802097fba50 ffffffff81d486b8 ffff8802097fba40 ffffffff810810f2
[ 1331.845236]  ffff88010c30c0c0 0000000000000003 ffff88017a991800 0000000000000000
[ 1331.845242] Call Trace:
[ 1331.845255]  [<ffffffff813f13b3>] dump_stack+0x63/0x90
[ 1331.845263]  [<ffffffff810810f2>] warn_slowpath_common+0x82/0xc0
[ 1331.845269]  [<ffffffff8108118c>] warn_slowpath_fmt+0x5c/0x80
[ 1331.845284]  [<ffffffff81618802>] usb_submit_urb.part.6+0x142/0x560
[ 1331.845289]  [<ffffffff81618c82>] usb_submit_urb+0x62/0x70
[ 1331.845297]  [<ffffffffc073020b>] acm_submit_read_urb+0x3b/0x80 [cdc_acm]
[ 1331.845304]  [<ffffffffc07330b6>] acm_port_activate+0x186/0x1e8 [cdc_acm]
[ 1331.845310]  [<ffffffff814f3a7e>] tty_port_open+0x8e/0xe0
[ 1331.845317]  [<ffffffffc0730d03>] acm_tty_open+0x33/0x60 [cdc_acm]
[ 1331.845324]  [<ffffffff814ebfff>] tty_open+0x11f/0x6b0
[ 1331.845331]  [<ffffffff81211b8f>] chrdev_open+0xbf/0x1b0
[ 1331.845337]  [<ffffffff8120acef>] do_dentry_open+0x1ff/0x310
[ 1331.845341]  [<ffffffff81211ad0>] ? cdev_put+0x30/0x30
[ 1331.845346]  [<ffffffff8120be84>] vfs_open+0x54/0x80
[ 1331.845352]  [<ffffffff81217aeb>] ? may_open+0x5b/0xf0
[ 1331.845358]  [<ffffffff8121b667>] path_openat+0x1b7/0x1330
[ 1331.845365]  [<ffffffff8121d9d1>] do_filp_open+0x91/0x100
[ 1331.845371]  [<ffffffff8122b266>] ? __alloc_fd+0x46/0x190
[ 1331.845377]  [<ffffffff8120c258>] do_sys_open+0x138/0x2a0
[ 1331.845383]  [<ffffffff8106b544>] ? __do_page_fault+0x1b4/0x400
[ 1331.845389]  [<ffffffff8120c3de>] SyS_open+0x1e/0x20
[ 1331.845395]  [<ffffffff8182dfb2>] entry_SYSCALL_64_fastpath+0x16/0x71
[ 1331.845400] ---[ end trace 53721cd36948f595 ]---

[me-no-dev]

@cnlohr please feel free to forward the zip to @leopck. Since we are talking about devices that actually break specs, it could be that the kernel is patched, though I doubt it on linux.

[me-no-dev]

@cnlohr about "code has to execute /very/ fast!": you should be able to NACK a couple of times and then give the answer. Maybe you can use a queue for responses and send those or NACK if none.

[cnlohr]

Actually, Micah in a different thread on Twitter pointed out you can pre-emptively ACK as well, but, based on my traces that might not be the best move. At low-speed I think we'll have time, just not to go and do a ton of stuff. EDIT: zip was forwarded to @leopck

[atc1441]

I know it's a dead thread but i would be very nice if you guys still working on this. I tried to port it myself but it's too hard for the first time of doing this

[devyte]

@cnlohr your README says that SDK >= 2.0 are incompatible. We (Arduino) are currently at SDK 2.2+. What is the reason for the incompatibility?

[cnlohr]

The reason originally was because the new SDKs use up precious IRAM. Espressif indicated this was something they would consider reducing in the future. I have not tried a new SDK to see if they have reduced their IRAM footprint. If they reduced it, then it will work A-ok!

[SpiraMirabilis]

The newest SDK still uses too much RAM. Emulating USB via software is right on the edge of the capabilities of this device. I've switched to ESP32 for some projects, although the disparity in cost is a bit of a pain. ESP8285 is my favorite price point/size/speed microcontroller of all time, though.

On Wed, Feb 28, 2018 at 12:44 PM, CNLohr notifications@github.com wrote:

The reason originally was because the new SDKs use up precious IRAM. Espressif indicated this was something they would consider reducing in the future. I have not tried a new SDK to see if they have reduced their IRAM footprint. If they reduced it, then it will work A-ok!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cnlohr/espusb/issues/1#issuecomment-369359488, or mute the thread https://github.com/notifications/unsubscribe-auth/ABf2gLnj0OdxOHHGS0ghDFAiD1V5y_mLks5tZayogaJpZM4Jewxs .

[cnlohr]

It's so frustrating cause it would be so easy for them to move more stuff out of IRAM... Even an extra 250 bytes would make it so much easier.

@SpiraMirabilis How much is it overflowing for you?

[devyte]

@cnlohr I took a look at the code in this repo, and it looks rather alien to me. Can we discuss directly? I'm on gitter.

[cnlohr]

I'm not really gonna have a ton of time to dig into this for a bit :(

[devyte]

@cnlohr have you gotten back to this at all? A lot has changed since I last looked, including:

• an effort to free up IRAM (maybe sdk2.x is now viable?)
• a precache tool which preloads a function into the cache so that during its execution there is no fetching from flash. In certain cases, this allows not using IRAM for that function.

[cnlohr]

I have not examined this at all. Would be cool if someone else could take this and run.

[devyte]

@cnlohr well, I've been looking for a while now for someone to do that, but I haven't found anyone who understands both usb and asm, and also has the interest and availability to pursue. I'd love to see your work here be wrapped into a library in our Arduino core and have e.g. flashing over this usb interface and allow boards that don't have the serial2usb chip at all. I'll continue looking for someone if you can't continue this work at this time.

[cnlohr]

@devyte honestly, you shouldn't need to touch any of the ASM... You WILL need to know some basic USB, especially to make apps on the host side to talk to the things. Maybe there's something I can help with. You can contact me directly if you'd like an easier back-forth support.