sclone fails due to SSL cert declined

[maphew]

Environment: Win10 with Windows Subsystem for Linux v2 (WSL2) running Arch Linux:

[matt@geolaptop mypdq]$ jsish -m sclone
current directory is not within an open checkout
clone https://jsish.org/afossil/pdq mypdq.fossil

Long pause, nothing happenng. Pressed enter.

SSL cert declined
server returned an error - clone aborted
CALL BACKTRACE:
#1: Manage.jsi:348:  in moduleRun( "sclone" )
#2: Manage.jsi:345:  in Manage( [ "sclone" ], {} )
#3: Jsish.jsi:344:  in Jsi_CmdTable( [ "sclone" ], { createmod:[ "function createmod(fn:string, tmpname:string=\"Mo ...)
#4: Manage.jsi:80:  in sclone()
#5: Manage.jsi:141:  in fossil( "clone https://jsish.org/afossil/pdq mypdq.fossil" )

/zvfs/lib/Manage.jsi:141: error: program exit code (1)
ERROR:

After much flailing about ran fossil clone https://jsish.org/afossil/pdq mypdq.fossil manually and realized the long pause is due to hidden prompts, the default to which are No. There are 3 such prompts. I can't get a record of them now as my machine has cached them, but they are along the lines of

1. SSL cert hash is ABCD1231EF456, do you accept [y,N]?
2. remember this for later [y,N]?
3. mumble autosync something [y,N]?

So running again and this time pressing Y-Enter for each pause:

jsish -m sclone
current directory is not within an open checkout
clone https://jsish.org/afossil/pdq code.fossil
Round-trips: 2   Artifacts sent: 0  received: 108
Clone done, wire bytes sent: 535  received: 3497792  ip: 50.116.0.90
Rebuilding repository meta-data...
  100.0% complete...
Extra delta compression...
Vacuuming the database...
project-id: 5b7e1585db8601042a0bf504a57c9a799ac8a12f
server-id:  4d471f06a9a4518e4b17e5abd510169d34d0d101
admin-user: matt (password is "adIUsjkbdi")

y
y
Autosync failed.
y
DONE! To finish setup run: "jsish ."

After this jsish . is successful. (Sort of. there are problems with wsl<->windows network filtering, not PDQ's problem. I'll write up more on that somewhere else later once I've learned more.)

[pcmacdon]

Short answer: Remove WSL, get Virtualbox and install Ubuntu 20.04.

Long answer: I recently upgraded my 10yr old laptop, and was commited to switching to Windows WSL, to replace Ubuntu as my desktop.

When I found out WSL allows no gui, I tried WSL2. I really wanted it to work, but after much thrashing around it became clear that WSL is infeasible for real work, and is suitable only for very light-duty Linux usage. VBox on the other hand lets Windows desktop users actually do real work in Linux, without constantly getting in the way.

One of the many issues with WSL (which I won't recount here) is it allows no other VMs. So PDQ will likely never support it.

FYI, make sure you upgrade to fossil 2.16, as I mentioned here: https://github.com/pcmacdon/jsish/discussions/45 and https://github.com/pcmacdon/jsish/discussions/47

[maphew]

Thanks for the WSL background info. I'm new into exploring it so this is a good time to change course in favour of virtual box if it's just not ready yet. I've been happy with vbox before and am/was exploring wsl with the idea it would be more performant since it's coming from the makers of Windows.

I think this is a side issue to the hidden SSL cert prompts though.

[pcmacdon]

Same here. I used VBox-linux in corporate managed Windows PCs for 7 years: it was mostly satisfactory. But there were occasional niggles so I figured WSL (based on the reviews I read) would be better: lower resource overhead, better response time, etc.

Boy was I wrong. There turned out to be an endless litany of things that "just don't quite work the way you expect", such as your SSL prompts for example. In hindsight I guess I shouldn't be surprised. MS has their own objectives, and they do not always align with ours.

I should also admit that I am now back using a native Ubuntu install. And for the first time in more than a decade I am using the default desktop (instead of alternatives like Lubuntu or Xubuntu). I find it is quite good, once you reconfigure it to look/act more like legacy Linux desktops.

[pcmacdon]

FYI, for your SSL problem I did the following:

$:~/tmp/pdq_4$ strings /usr/local/bin/fossil | grep SSL | grep -wi accept
$:~/tmp/pdq_4$ strings /usr/local/bin/fossil | grep SSL | grep -wi cert
SSL cert declined
Unable to verify SSL cert from %s
OpenSSL-cert-file: %s
OpenSSL-cert-dir:  %s
$:~/tmp/pdq_4$ 

I see "SSL cert declined" but not seeing the other messages. So they must be coming from the os (Arch):

BTW: what fossil version are you using.

[maphew]

my first attempt was with the fossil version served up from jsish.org, which was 2.10 I believe. I then used: 2.17 [1e5dc324ef] 2021-09-13 21:30:15 UTC

I just tried to clone a fossil repo of mine from within the Arch machine and there were no messages at all. I tried blindly usingY-Enter as above to o effect. After breaking with ctrl-c there is an x.fossil and x.fossil-journal which fossil info reports is project and code . ...so yeah, this vm is broken in ways I don't care to troubleshoot.

[pcmacdon]

It does seem that WSL is Microsoft's answer to Wine. And works about as well.

Thanks for the heads up, I've updated the jsish.org/download/fossil binaries to 2.16.

On Sat, 18 Sept 2021 at 22:38, matt wilkie @.***> wrote:

my first attempt was with the fossil version served up from jsish.org, which was 2.10 I believe. I then used: 2.17 [1e5dc324ef] 2021-09-13 21:30:15 UTC

I just tried to clone a fossil repo of mine from within the Arch machine and there were no messages at all. I tried blindly usingY-Enter as above to o effect. After breaking with ctrl-c there is an x.fossil and x.fossil-journal which fossil info reports is project and code . ...so yeah, this vm is broken in ways I don't care to troubleshoot.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pcmacdon/pdq/issues/2#issuecomment-922418479, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE3J7BQ6PLJSH3VNK3TR6BLUCVZMTANCNFSM5EHYA3AQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[maphew]

One more data point before I lay wsl arch down: fossil ui from within the mypdq folder works. Accessing http://localhost:8080 from Windows browser shows the expected fossil pages, and everything seems functional. Thinking it might be port filtering I turned Windows Firewall off for all networks and tried jsish . again but still wasn't able to connect.

[pcmacdon]

Did you try jsish . -port 8088? Otherwise jsish/pdq will use a random port. I think this may be one of the things the doesn't work properly under MS.

[maphew]

I have now, and no observable change.

[pcmacdon]

To clarify, I should have said use jsish . -port 8080 ie. after shutting down fossil ui

I don't really know how WSL networking works. Does it really let windows host access guest PORTS using 'localhost'?

I don't see how this can work unless you setup a port proxy on Windows. Or is that what you've done?

[pcmacdon]

Ok, according to a Stackoverflow post:

WSL2 exposes ports on the local interface (which is why in Windows you can access localhost:8080 when your 8080 service is running in WSL2), but they listen on 127.0.0.1

That answers that question.

As to why jsish may not work via WSL, I don't know. I don't have access to WSL now, as I gave up on it due to all the obstacles it introduced.

[maphew]

for future reference: https://docs.microsoft.com/en-us/windows/wsl/compare-versions#accessing-linux-networking-apps-from-windows-localhost

I have a working Kubuntu 2020 vm now. PDQ installed and running with minimum of fuss. It looks wonderfully feature full. No cert notices at all during clone stage.

So this is likely a Windows specific issue. I had a problem with fossil and SSL certs earlier this summer that I should have realised is probably related: https://fossil-scm.org/forum/forumpost/4dcd2f16c289848c.

[maphew]

I've got a new-to-me freshly installed Windows 10 machine. While waiting for new vbox setup to complete I re-ran the clone procedure in windows shell. I'm adding this here for the record and possible future investigation, not because I'm looking for attention on it now. It's clear that at this stage of PDQ it's better to focus on within-linux efforts.

---

fossil: version 2.16 [7aedd56758] 2021-07-02 12:46:01 UTC jsish: 3.5.0 3.05 d8b98ba5ea7ebcb8916c099efd5796281240a9a1 2021-09-15 21:49:49 UTC

I've renamed jsish.exe to jsi.exe. The '$' prompt instead of 'C:>' is from using ConEmu as default console host.

Part 1

$ jsi -m sclone
current directory is not within an open checkout
clone https://jsish.org/afossil/pdq mypqd.fossil
y
y
server did not reply
server returned an error - clone aborted
Unable to verify SSL cert from jsish.org
 subject: CN = jsish.org
 issuer: C = US, O = Let's Encrypt, CN = R3
 sha256: de057f98365f6a987ed5746daa78b15548b21e5e96536d92aaedced5371a9816
accept this cert and continue (y/N)? remember this exception (y/N)? Clone done, sent: 261 received: 0 ip: 50.116.0.90

CALL BACKTRACE:
#1: Manage.jsi:352: in moduleRun( "sclone" )
#2: Manage.jsi:349: in Manage( [ "sclone" ], {} )
#3: Jsish.jsi:344: in Jsi_CmdTable( [ "sclone" ], { createmod:[ "function createmod(fn:string, tmpname:string=\"Mo ...)
#4: Manage.jsi:85: in sclone()

Manage.jsi:85: error: sqlite error: no such table: config in statement at line 1  (c-extn [Sqlite])
ERROR:

Resulted in 0 byte mypdq.fossil file and nothing else.

Part 2

$ del mypqd.fossil

Matt@MK-DESKTOP C:\Users\Matt\code\mypqd
$ jsi -m sclone
current directory is not within an open checkout
clone https://jsish.org/afossil/pdq mypqd.fossil
Round-trips: 2  Artifacts sent: 0 received: 110
Clone done, sent: 536 received: 3498369 ip: 50.116.0.90
Rebuilding repository meta-data...
 100.0% complete...
Extra delta compression...
Vacuuming the database...
project-id: 5b7e1585db8601042a0bf504a57c9a799ac8a12f
server-id: 1ec0fd9e155108f4804eb5203852d77f2dba34d0
admin-user: Matt (password is "gXSnaoBZKC")

DONE! To finish setup run: "jsish ."

$ jsi .
FAI /var/www/fossil/pdq-plugins.fossil
directory C:/Users/Matt/code/mypqd is not empty
use the -f or --force option to override
CALL BACKTRACE:
#1: main.jsi:15: in moduleRun()
#2: pdq.jsi:776: in pdq( [], {} )
#3: pdq.jsi:687: in main()
#4: pdq.jsi:520: in sysInit()

C:/Users/Matt/code/mypqd/pdq/pdq.jsi:520: error: operand value to + is undefined
ERROR:

---

I suspect the errors in Part 1 would not have shown up if my first step had been to fossil clone http://jsish.org in order to get the background OpenSSL cert and fingerprint cached locally. Part 2 now repeats the same everytime (e.g. in a new empty mypdq2 folder).

[pcmacdon]

I've added to README.md the following: NOTE: currently works under native Linux, but not Windows nor WSL.

However you peaked my curiosity about Windows so I fired up my Windows 7 vm, and tried it.
Part 1 failed as you described, and yes you are correct if you do the fossil clone first no cert messages appear (yes, I tried on a different machine).

For Part 2 I've committed a fix for the bug you are seeing, if you'd like to try it again. However for me Part 2 fails differently and more completely (see below).
Meaning for now supporting Windows is likely unrealistic.

PART 2 FAILURE:

Z:\mypdq>jsish -m sclone
current directory is not within an open checkout
clone https://jsish.org/afossil/pdq mypdq.fossil
Round-trips: 2   Artifacts sent: 0  received: 110
Clone done, wire bytes sent: 537  received: 3498369  ip: 50.116.0.90
Rebuilding repository meta-data...
  100.0% complete...
Extra delta compression...
Vacuuming the database...
project-id: 5b7e1585db8601042a0bf504a57c9a799ac8a12f
server-id:  53c8d246d60c9f5207592125c31f97b44d108325
admin-user: pete (password is "AWpo4EUR2k")

SSL cert declined
Autosync failed.

unable to auto-sync the repository
use --repository or -R to specify the repository database
current directory is not within an open checkout
DONE! Change password with: fossil user password pete NEWPASS