ntadmin / DGND3700v2

Custom firmware for the Netgear DGND3700v2
12 stars 4 forks source link

Unable to mount ext3 USB disk #47

Closed vezza closed 7 years ago

vezza commented 7 years ago

I'd like to share an external usb ext3 hdd over my network... unfortunately It doesn't work, well the hdd itself work just fine on my PC so I'm preatty sure that it is a software problem on the router. Also I found the following discussion:

(https://community.netgear.com/ejquo23388/board/crawl_message?board.id=home-dsl-modems-routers&message.id=8839)

that outlines what the problem is and how to solve it... anyway I'm not a programmer so I'm not able to fully understand/fix the issue. Thanks for any help and/or reply!

ntadmin commented 7 years ago

I have sent a request out on that forum for the script that is mentioned. If I can get the script I'll look into adding it to the custom firmware.

vezza commented 7 years ago

Thanks :-)

vezza commented 7 years ago

I think the above linked discussion is dead... anyway I've managed to manually mount the usb device, so I guess the problem resides in the scripts that are executed when a usb device is attached. I'll look in to this so to find a workaround... The following output may help to point me in the right direction (what do you think about the sh: /usr/sbin/rc_app/rc_apple: not found error message?)

/sbin/rc_app/rc_usb_service mount SHARES_ALL: sd list:

Disk X: volume_name=U Drive device_name=0A capacity=312569856 serial_num=D:2 type= dev_node=sda2 approved=1

Shere List:: Share Name=USB_Storage Folder= R=1 W=1 list done.

username=admin,passwd=**** Added user admin. sh: /usr/sbin/rc_app/rc_apple: not found killall: minidlna: no process killed PATH_PREFIX is empty now open MEDIA_CFG_FILE Segmentation fault`

ntadmin commented 7 years ago

@vezza, I'm not at home much for the next month, when I am back I engage again with the question. It would be good for the firmware to handle ext3.

The basic methodology of replacing rc_usb_service with a version which (also) handles ext3 seems sensible to me, I look forwards to hearing your suggestions.

My guess is that the rc_apple thing is probably an attempt to start apple style file sharing which isn't actually implemented.

On 1 Aug 2017, at 22:31, vezza notifications@github.com wrote:

I think the above linked discussion is dead... anyway I've managed to manually mount the usb device, so I guess the problem resides in the scripts that are executed when a usb device is attached. I'll look in to this so to find a workaround... The following output may help to point me in the right direction (what do you think about the sh: /usr/sbin/rc_app/rc_apple: not found error message?)

/sbin/rc_app/rc_usb_service mount SHARES_ALL: sd list:

Disk X: volume_name=U Drive device_name=0A capacity=312569856 serial_num=D:2 type= dev_node=sda2 approved=1

Shere List:: Share Name=USB_Storage Folder= R=1 W=1 list done.

username=admin,passwd=**** Added user admin. sh: /usr/sbin/rc_app/rc_apple: not found killall: minidlna: no process killed PATH_PREFIX is empty now open MEDIA_CFG_FILE Segmentation fault`

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

vezza commented 7 years ago

@ntadmin, Ok, don't mind I've been away from home too... Now I'm back, I hope I can help, thanks for your work

ntadmin commented 7 years ago

OK, I'm back now, but #48 is current priority as it's a key feature failure. Once I've got that zapped I'll get onto your request. Feel free to fork the source if you wish to ...

ntadmin commented 7 years ago

I've tried a direct message to the person who wrote the script on the off chance it gets through, both via the netgear forum and xda-developers where it looks they also have been. I'll give them a couple of days or so. If they don't reply I'll try and delve deeper into the process they describe, but I will need help with testing!

vezza commented 7 years ago

I'm happy to help with testing... unfortunately I'm not so good in coding, sorry

ntadmin commented 7 years ago

Cool. This thread will start accumulating information learnt along the way, and I'll flag when a alpha becomes available!

ntadmin commented 7 years ago

When you insert a usb drive, the sequence of rc calls is as follows:

Mon Sep  4 12:15:02 BST 2017 [rc_apps] /usr/sbin/rc_app/rc_usb_service kmount sda Card  Reader     0:0:0:0 1
Mon Sep  4 12:15:06 BST 2017 [rc_apps] /usr/sbin/rc_app/rc_smb event mount
Mon Sep  4 12:15:11 BST 2017 [rc_apps] /usr/sbin/rc_app/rc_ftpd event mount
Mon Sep  4 12:15:11 BST 2017 [rc_apps] /usr/sbin/rc_app/rc_smb_http_en_chk event mount
Mon Sep  4 12:15:11 BST 2017 [rc_apps] /usr/sbin/rc_app/rc_mediaserver event mount

Outcomes:

So, if we can make rc_usb_service mount an ext3 device on /mnt/shares/U, we should be on the right track.

ntadmin commented 7 years ago

Now, let's put an ext3 drive in. It fails, although it still runs all of the scripts, although it fails in the first one, as the above would suggest. So, it runs:

/usr/sbin/rc_app/rc_usb_service kmount sda Card Reader 1:0:0:0 1

If we run that, the output is:

node=sda
mknod /dev/sda1
mknod /dev/sda2
mknod /dev/sda3
mknod /dev/sda4
mknod /dev/sda5
mknod /dev/sda6
mknod /dev/sda7
mknod /dev/sda8
mknod /dev/sda9
mknod /dev/sda10
mknod /dev/sda11
mknod /dev/sda12
mknod /dev/sda13
mknod /dev/sda14
mknod /dev/sda15
mknod /dev/sda16
SHARES_ALL:
sd list:

 Disk X:
  volume_name=U Drive
  device_name=Card
  capacity=983024
  serial_num=
  type=FAT
  dev_node=xxx
  approved=0

 Shere List::
  Share Name=USB_Storage  Folder=<root> R=1 W=1

 Disk X:
  volume_name=U Drive
  device_name=Card
  capacity=983024
  serial_num=
  type=FAT
  dev_node=sda1
  approved=0

 Shere List::
  Share Name=USB_Storage  Folder=<root> R=1 W=1
list done.

init C: non DOS media
Cannot initialize 'C:'
/usr/sbin/mlabel: Cannot initialize drive
mount: mounting /dev/sda1 on /mnt/shares/U failed: Invalid argument
sh: /usr/sbin/rc_app/rc_apple: not found
killall: minidlna: no process killed
#### media server is disable

So, it manages to work out that it is /dev/sda1 through this process, but not that the type is ext3, not FAT.

ntadmin commented 7 years ago

Now, the version of mount on the system is pretty clever. Whether you insert an ext3 drive of a FAT drive, umount it (as it will have been auto mounted or not by the netgear code), if you type:

mount /dev/sda1 /mnt/shares/U

it will mount the thing. So, fairly easy to knock up a script to replace rc_usb_service that finds the device and mounts it. However, the next rc isn't then called, so the first rc script must call the next four.

ntadmin commented 7 years ago

@vezza I have now got a version which might work, are you able to edit files on your router? If so, I can tell you what to do to test, and we can check fairly quickly if it works.

vezza commented 7 years ago

@ntadmin Yes I can edit files on my router... please tell me what to do and I'll try

ntadmin commented 7 years ago

@vezza, cool. In theory it should only take two things. If you could do these and let me know that would be fantastic. On the router:

  1. Remove /usr/sbin/rc_app/rc_usb_service and replace it with the file here: https://github.com/ntadmin/DGND3700v2/blob/master/rootweb/additions/usr/sbin/rc_app/rc_usb_service
  2. chmod a+x /usr/sbin/rc_app/rc_usb_service
  3. Remove /etc/samba.conf/smb.conf and replace it with the file here: https://github.com/ntadmin/DGND3700v2/blob/master/rootweb/additions/usr/etc/samba.conf/smb.conf
vezza commented 7 years ago

@ntadmin So, now the usb disk has been mounted but it doesn't show up on the adv_index.htm (Storage USB) page on the router's gui... Anyway very good job :+1:

ntadmin commented 7 years ago

@vezza cool, this is a good start. A couple of questions if I may.

  1. When the disk is inserted and mounted, is it available as an export. I.e. Can you see it from another computer? It may be hat this bit is working even if the config page isnt't.

  2. When the disk is inserted and mounted, could you post the contents of /etc/samba/smb.conf please. This is different to the file which you changed, which was in /etc/samba.conf ... somewhat odd I realise ....

This will allow me to check if the things I think the code now does are being done, once we know that, we can work out what to do next, although I'm now into a busy few days at work, so please don't worry if I'm silent for a while ...

Thanks,

vezza commented 7 years ago

@ntadmin :

1) Yes I can mount the disk from other PC;

2) Here is my /etc/samba/smb.conf:

# cat /etc/samba/smb.conf
[global]
        config file=/etc/samba/smb.conf
        display charset = utf8
        unix charset = utf8
        dos charset = utf8
        ;os level = 8
        workgroup =Workgroup
        server string =readyshare
        netbios name =readyshare
        netbios aliases =
        load printers = no
        disable spoolss = yes
        max log size = 10
        security = user
        encrypt passwords = yes
        smb passwd file = /etc/samba/smbpasswd
        socket options = TCP_NODELAY SO_KEEPALIVE SO_SNDBUF=65536 SO_RCVBUF=65536
        ;preferred master = yes
        dns proxy = no
        preserve case = yes
        short preserve case = yes
        default case = upper
        case sensitive = yes
        mangled names = yes
        null passwords = yes
        dos filetimes = yes
        kernel oplocks = no
        veto files = /.ShareConfFile/quota.user/quota.user~/$AttrDef/$BadClus/$Bitmap/$Boot/$Extend/$LogFile/$MFT/$MFTMirr/$Secure/$UpCase/$Volume/lost+found/System Volume Information/
        delete veto files = False
        force directory mode=771
        force create mode=660
        create mask=771
        ;if set map system as true, it would cause file treat as executable file
        map system=no
        syslog=0
        log file=/dev/null
        guest ok=yes
        map to guest=Bad User
        guest account=nobody
        include=/etc/samba/smb.usb.conf

Don't worry I'm quite busy too...

ntadmin commented 7 years ago

OK, so we have it working, now we just have to make the interface reflect the fact that it is working!! I'll ponder that as a next challenge!

vezza commented 7 years ago

@ntadmin:

1) Please have a look at this: http://www.evolware.org/?p=114, don't know if it may be useful...; 2) Is this correct?

in /etc/samba/smb.conf:

...
workgroup =Workgroup
server string =readyshare
netbios name =readyshare
netbios aliases =
...

in /etc/samba.conf/smb.conf:

...
workgroup = DG834GVU
server string = Sercomm
netbios name = Sercomm DG834GVU
netbios aliases = Sercomm
...
ntadmin commented 7 years ago

@vezza

I think that the system works in the follow way:

The change I have made is to add a line to the reference file (/etc/samba.conf/smb.conf) requiring to include another file (/etc/samba/smb.usb.conf). That file is created on the fly by the updated rc_usb_service code to tell the samba server to serve the USB drive.

Hope that makes sense.

Whether you consider the way the Netgear code approaches this to be correct or not I leave to you :-)

vezza commented 7 years ago

@ntadmin It makes sense... don't know if it is correct anyway. So... if now we have samba shares... why they doesn't show up on the router's gui? Well, in the Home section of the index.htm page the ReadyShare Status is reported to be Ready but in the Readyshare section of the same page nothing is getting showed...

vezza commented 7 years ago

@ntadmin In order to understand a little bit more (hopefully to help you a little) I was searching some documentation... I found this netgear site: https://kb.netgear.com/2649/NETGEAR-Open-Source-Code-for-Programmers-GPL Do you think it may be usefull or not? Also could it be interesting for the Openwrt project? Maybe they could add support for this router?

vezza commented 7 years ago

@ntadmin Sorry, I now see that you are also on the Openwrt forum so maybe you already know everything I am telling you...

ntadmin commented 7 years ago

@vezza I appreciate all the help anyone is willing to chip in, thanks for helping out!

The alternative firmware on this project is based on the source code from Netgear, which unfortunately doesn't have source code for some key executables, so we have to infer how they handle situations when we wish to improve their behaviour!

I think that the reason that the web page doesn't show the drive, even though it can be seen on the network, is that the alternative rc_usb_service I have produced does the recognition and samba serving bit, but it doesn't update the internal information stack so that it show correctly on that page. That's the next step!

I think that I have identified the internal information variables, and I have a hunch as to their format. I'll post more information when I next have time to look in detail but if you do

grep USB /tmp/nvram

on the router you will probably (not certainly) see them listed, with entries left over from previous mounts, done before you made the file changes.

vezza commented 7 years ago

@ntadmin Yes you are right:

# grep USB /tmp/nvram
storage_share_def=U:USB_Storage:-:0:0T:T_Drive:-:0:0S:S_Drive:-:0:0R:R_Drive:-:0:0
sd_share=0AU Drive312569856D:21XxxxUSB_Storage11
ntadmin commented 7 years ago

After a correctly mounted FAT disk, having reset both storage_share_def and sd_share, only sd_share is set. It is set using their array format, one line, as follows (':' is cell separator)

Card:U Drive:983024::FAT:0:U:sda1:USB_Storage:1:1 After removal, the sda1 is changed to xxx, the U becomes X, everything else remains the same.

ntadmin commented 7 years ago

@vezza Could you please run:

nvram-util examine sd_share nvram-util examine storage_share

and post your output please? It's a bit long, but it's a detailed examination of everything in those variables, and I have messed with mine too much to be sure I am seeing the netgear patterns. Thanks!

vezza commented 7 years ago

@ntadmin here's the outputs:

# nvram-util examine sd_share
[  0] '0' 048
[  1] 'A' 065
[  2] '@' 002
[  3] 'U' 085
[  4] ' ' 032
[  5] 'D' 068
[  6] 'r' 114
[  7] 'i' 105
[  8] 'v' 118
[  9] 'e' 101
[ 10] '@' 002
[ 11] '3' 051
[ 12] '1' 049
[ 13] '2' 050
[ 14] '5' 053
[ 15] '6' 054
[ 16] '9' 057
[ 17] '8' 056
[ 18] '5' 053
[ 19] '6' 054
[ 20] '@' 002
[ 21] 'D' 068
[ 22] ':' 058
[ 23] '2' 050
[ 24] '@' 002
[ 25] '@' 002
[ 26] '1' 049
[ 27] '@' 002
[ 28] 'X' 088
[ 29] '@' 002
[ 30] 'x' 120
[ 31] 'x' 120
[ 32] 'x' 120
[ 33] '@' 003
[ 34] '@' 002
[ 35] 'U' 085
[ 36] 'S' 083
[ 37] 'B' 066
[ 38] '_' 095
[ 39] 'S' 083
[ 40] 't' 116
[ 41] 'o' 111
[ 42] 'r' 114
[ 43] 'a' 097
[ 44] 'g' 103
[ 45] 'e' 101
[ 46] '@' 002
[ 47] '1' 049
[ 48] '@' 002
[ 49] '1' 049
[ 50] '@' 003
[ 51] '@' 001
# nvram-util examine storage_share
#
vezza commented 7 years ago

nvram-util examine storage_share has no output...

ntadmin commented 7 years ago

Thanks, all very interesting!

In case you are interested, in the 'array' format they use marks the end of a cell with \002 and the end of a row with \001. What the two \003 characters represent is not yet clear!

vezza commented 7 years ago

maybe \003 code is: https://en.wikipedia.org/wiki/End-of-Text_character

ntadmin commented 7 years ago

For the record, using netgear's rc_usb_sercice, here is sd_share:

[  0] 'C' 067
[  1] 'a' 097
[  2] 'r' 114
[  3] 'd' 100
[  4] '@' 002
[  5] 'U' 085
[  6] ' ' 032
[  7] 'D' 068
[  8] 'r' 114
[  9] 'i' 105
[ 10] 'v' 118
[ 11] 'e' 101
[ 12] '@' 002
[ 13] '9' 057
[ 14] '8' 056
[ 15] '3' 051
[ 16] '0' 048
[ 17] '2' 050
[ 18] '4' 052
[ 19] '@' 002
[ 20] '@' 002
[ 21] 'F' 070
[ 22] 'A' 065
[ 23] 'T' 084
[ 24] '@' 002
[ 25] '0' 048
[ 26] '@' 002
[ 27] 'U' 085
[ 28] '@' 002
[ 29] 's' 115
[ 30] 'd' 100
[ 31] 'a' 097
[ 32] '1' 049
[ 33] '@' 003
[ 34] '@' 002
[ 35] 'U' 085
[ 36] 'S' 083
[ 37] 'B' 066
[ 38] '_' 095
[ 39] 'S' 083
[ 40] 't' 116
[ 41] 'o' 111
[ 42] 'r' 114
[ 43] 'a' 097
[ 44] 'g' 103
[ 45] 'e' 101
[ 46] '@' 002
[ 47] '1' 049
[ 48] '@' 002
[ 49] '1' 049
[ 50] '@' 003
[ 51] '@' 001
ntadmin commented 7 years ago

It looks like this is the only place that \003 turns up in all of their stored variables!

vezza commented 7 years ago

I realized that I can only connect to the samba share from other PC as admin user... it seems that user nobody doesn't work...

# cat /etc/samba/smbpasswd 
nobody:501:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:[DU         ]:LCT-00000000:
admin:0:85A1C8AF54AD43C525AD3B83FA6627C7:C78FCD44AD86E4F070925950B80EDABE:[U          ]:LCT-59B2BDD7:
ntadmin commented 7 years ago

I've updated the rc_usb_service so that the drive should now appear on the list. I'll get onto the mounting issue next!

vezza commented 7 years ago

@ntadmin I can confirm that now the drive is getting showed on the list :+1:

vezza commented 7 years ago

@ntadmin Now the drive is disappeared from "USB Drive Approved Devices" (see attached screenshot)

anyway the nobody user problem is fixed (I can now browse the share as nobody user)

schermata da 2017-09-10 11-39-32

ntadmin commented 7 years ago

Hmmm, we seem to have inconsistency here. Out of interest, did you reboot the router between when it did work and when it didn't?

vezza commented 7 years ago

@ntadmin yes, the router did a reboot because of a power failure...

ntadmin commented 7 years ago

OK, that would explain it - you will find that it has reverted /etc/samba.conf to the default version ... easy to solve by copying the updated smb.conf back in (if you want to avoid this in the future, copy the updated version also to /usr/stc/samba.conf ...) In a full firmware release this will be dealt with :-)

vezza commented 7 years ago

Today I haven't been able to fix the missing approved device issue, certainly I'm doing something wrong... will try again tomorrow

ntadmin commented 7 years ago

I hope to put up an alpha release (17A2) with some of these things sorted, when I have checked it on my router. This will make checking and moving forwards easier because all the changes so far will be in it!

ntadmin commented 7 years ago

OK, I've put out an alpha release which I think gathers together various of the resolved bugs and some general mainenance work - NEWT17A2.

@vezza - would you be able to install and check. Please be aware of #42 and the work around (Remove the pppoe cable, wait a minute or two, then do the upgrade. I that fails, when it has rebooted, don't connect the pppoe cable and upgrade from there.)

vezza commented 7 years ago

@ntadmin of course... will report asap

vezza commented 7 years ago

@ntadmin - Now I'm running NEWT17A2 , everyting is running fine but I am still missing usb devices in the "Approved Devices" page

ntadmin commented 7 years ago

Good point. I had missed that one. To keep things simple, I'm going to mark this issue as closed and open two separate issues (#51 and #52) for the USB file sharing system.