[kohlchan.net]: Support discussion

sunabozu commented 5 years ago

Is your feature request related to a problem? After the recent switch to https://gitgud.io/Tjark/KohlNumbra Kohlchan doesn't support Dollchan anymore.

Describe the solution you'd like to see in the implementation Can somebody look into it and tell if it's possible to add the support?

StephenLynx commented 5 years ago

I'd just like to point that they are using lynxchan now. But they made a few changes to their front-end, so support for other lynxchan sites might not work with them.

SthephanShinkufag commented 5 years ago

The lynxchan engine is developing and now it has a new posting mechanism which conflicts with the Dollchan. Posting does not work on all lynxchan imageboards at the moment.

StephenLynx commented 5 years ago

Ah, you mean the api merge? Yeah, that changed on 2.2.

ganego commented 5 years ago

Things that do not work:

Youtube previews (Example: https://nocsp.kohlchan.net/tv/res/3758.html ). Despite the name, at the moment nocsp domain serves csp. But this is not the issue, as it also does not work with csp disabled in firefox). EDIT: This is because YT links are plain text instead of links. So VideosParser._parserHelper('a[href*="youtu"]',... won't work. Admin does not care and won't change this to clickable links. EDIT: Can be fixed by setting this.hasTextLinks = true; for lynxchan (or kohlchan).
~~Tab bar: New post warning will not be removed after viewing the tab (green number & favicon).~~ EDIT: Fixed, see below.
"Post menu" button (no idea what this is for).
~~Password has to be 8 or less digits. Maybe admin will fix this.~~ EDIT: Fixed, now allows for longer passwords
Sometimes (due to some temporary 502 (bad gateway) or so) the favicon will change to the 404 one with the red X in it. The thread is still alive though. So upon the next autoupdate (that will occur when I switch to this tab) dollchan should check the favicon and if it's the 404 one, but it just got a proper update, the 404 icon should be removed). EDIT: Fixed by admin. favicon now sent with correct http header.

Thank you!

sunabozu commented 5 years ago

@ganego The last issue can be easily fixed by setting a shorter password in the settings. But after that you get a new problem:

This board requires at least one file when creating threads.

It doesn't recognize the attached files.

ganego commented 5 years ago

Update: YT preview and links in general can be fixed by setting this.hasTextLinks = true;

ganego commented 5 years ago

Admin changed PW length. Now allows for more than 8 chars again. EDIT: Do not use more than 8 chars though. It will bug because KC simply cuts all after the first 8 chars.
Regarding favicons not changing when new posts were viewed: EDIT: FIXED

There is this code part (es6 version):

updateIcon(isError) {
    if(!isError && !newPosts) {
        this._setIcon(this.originalIcon);
    } else if(this._hasIcons) {
        this._setIcon(isError ? this._iconError :
            repliesToYou.size ? this._getIconYou(newPosts) : this._getIconNew(newPosts));
    }
}

Now if you add a console.log(this.originalIcon); before the first _setIcon it will say "https://kohlchan.net/favicon.ico" as expected. But setting the icon does not work. If I replace this.originalIcon with https://google.com/favicon.ico it works perfectly fine and after viewing the thread, the favicon in the tab bar will switch to the google favicon. Strange.

EDIT: When opening a thread for the first time, I don't have any favicon at all. Related? EDIT2: I guess there is something wrong with the favicon. Original one in my FF opens as garbled text, whereas the google one is a proper image. Will investigate further. EDIT3: Admin fixed favicon http headers. Favicons now work as expected.

ganego commented 5 years ago

Update: Favicon fixed by admin (was wrong http header).

ganego commented 5 years ago

So the only thing left not working is posting/with images. Posting itself works (without image), but dollchan won't realize this (post is successfully sent and shows up, when refreshing). Cannot really help there.
One difference between DC and lynxchan autoupdater: DC reloads the whole page, whereas lynxchan just fetches some much smaller json.

StephenLynx commented 5 years ago

https://gitgud.io/LynxChan/PenumbraLynx/blob/master/static/js/api.js#L303

ganego commented 5 years ago

So I pretty much spammed the source with console.log() to find out what actually happens (functions like html5Submit, sendHTML5Post and _initAjaxPosting (the this.form.onsubmit part), and, uhm, nothing happens?
I tried it on endchan and pretty much all my debug log() fired, whereas on KC nothing happened.

The first thing that should be triggered would be:

_initAjaxPosting() {
    let el;
    console.log("_initAjaxPosting")
    if(aib.qFormRedir && (el = $q(aib.qFormRedir, this.form))) {
        aib.disableRedirection(el);
        console.log("aib.disableRedirection")
    }
    console.log(this.form)
    this.form.onsubmit = e => {
        console.log("this.form.onsubmit - BEFORE")
        $pd(e);
        $popup('upload', Lng.sending[lang], true);
        html5Submit(this.form, this.subm, true).then(checkUpload)
            .catch(err => $popup('upload', getErrorMessage(err)));
        console.log("this.form.onsubmit - AFTER")
    };
}

The init works fine (console.log(this.form) gets printed upon page-load and is <form action="/replyThread.js" enctype="multipart/form-data" method="post" style="display: inline-block; text-align: left;">). But the event does not get triggered at all. On endchan it gets triggered (.log() prints something). Now why does this simply event not get triggered?

I saw one difference from endchan to kohlchan: document.getElementById("formButton") (which is this.subm) has two onClick events on KC and only the dollchan one on EC. But I also tested this by removing the whole #formButton and adding it again to remove the KC event listener:

init() of KC:

var ff = document.getElementById("formButton");
var hh = ff.outerHTML;
var par = ff.parentElement;
ff.parentElement.removeChild(ff.parentElement.children[2]);
par.innerHTML = par.innerHTML + hh;

This removed the eventListener, but it did not trigger the event listener anyway (and I found out, that the original onClick event seems to be the thing still making KC somewhat work).

The fuck?

StephenLynx commented 5 years ago

https://gitgud.io/LynxChan/PenumbraLynx/blob/master/static/js/api.js#L86 The method kc uses to enable js on posting is different from the one endchan uses.

ganego commented 5 years ago

Ok more progress. First of all remove the old button and make it "submit":

KC init:

init() {
    super.init();
    $script('thread.postReply = function() {}');
    $script('thread.startTimer = function(time) {}');

    var ff = document.getElementById("formButton");
    ff.type = "submit";
    var hh = ff.outerHTML;
    var par = ff.parentElement;
    ff.parentElement.removeChild(ff.parentElement.children[2]);
    par.innerHTML = par.innerHTML + hh;

    return false;
}

Now, fix the api URL:

In async sendHTML5Post(form, data, needProgress, hasFiles) { change it to return $ajax(form.action + '?json=1', ajaxParams). (Yes I'm aware this is hobopatching and will break Endchan, but this is mainly for info).

Next error to fix UnsupportedMediaTypeError: unsupported content-type. - Working on that. Basically change headers : { 'Content-Type': 'application/x-www-form-urlencoded' }, to headers : { 'Content-Type': 'multipart/form-data' }, but with some boundary.

@StephenLynx Why does KC not use application/x-www-form-urlencoded? At least this worked with Endchan.

SthephanShinkufag commented 5 years ago

@ganego, I'm already made all before what you are doing now, don't make the work twice.

SthephanShinkufag commented 5 years ago

The fix will be soon.

StephenLynx commented 5 years ago

@ganego because you have to use multipart to send files. Endchan uses the deprecated api that was just a big binary blob on the request body, while the new api is limited to the no-js input.

SthephanShinkufag commented 5 years ago

Seems the posting is working now. Please tell me some boards on new api with captcha, it need to be fixed.

SthephanShinkufag commented 5 years ago

Dollchan sends these parameters when old API:

captchaId  : cookieObj.captchaid,
bypassId   : cookieObj.bypass,
auth       : { login: cookieObj.login, hash: cookieObj.hash }

But for new API I ignored them. I need a board with captcha to test new API. I also ignored these parameters for files:

          form.append('fileMd5', file.md5);
          form.append('fileMime', file.mime);
          form.append('fileSpoiler', file.spoiler || '');
          form.append('fileName', file.name);

Everything works for now, maybe I didn't notice the problems yet.

ganego commented 5 years ago

EDIT: Forget the first question - me stupid.

Also: ajaxParams = { data, method: 'POST' };. This easy? Now I feel stupid for all the stuff I tried to get it to work (I tried to put dataObj manually in a string with correct boundaries for the multipart/form-data but it would always error with "BadRequestError: stream ended unexpectedly". Oh well, at least I learned new stuff :)

Thank you!

SthephanShinkufag commented 5 years ago

Heh. Sometimes decisions are much easier than they seem. I’m worried about fields that are generated by internal scripts. Bypass, captchaid, file.md5 and so on. Are they critically needed or not? And if so, which ones.

ganego commented 5 years ago

What does the bypass checkbox mean: Admin answer: "If checked, it checks for bypass first (request permission to post with Tor for example) and only if true sents the post with media to server." & "Checkbox is now removed. It didn't even worked like intended, so nothing missing."

The rest: No idea, let's wait for @StephenLynx

StephenLynx commented 5 years ago

The bypass check just checks if the user will require a block bypass to post before actually trying to post. It saves him time uploading files, mostly. All it does is to use the api on an entirely different operation.

Those parameters fileMd5 and others similars are used to reference files and spoil individual files. If you ignore that, users won't be able to post files without uploading them if they already exist on the server.

https://gitgud.io/LynxChan/PenumbraLynx/blob/master/static/js/thread.js#L604 How the check works.

https://gitgud.io/LynxChan/PenumbraLynx/blob/master/static/js/postCommon.js#L325 How the file processing works.

ganego commented 5 years ago

Password field in the delete form is not auto-filled.

kr4ssi commented 5 years ago

Images don't open in the viewer sometimes, because they have no extension, there's a data-attribute with the correct mime though. Can that be fixed?

SthephanShinkufag commented 5 years ago

@kr4ssi

Need a link to the long-live post with this kind of image.

ganego commented 5 years ago

Yeah I observed this bug some time ago too, but it's somewhat rare and I figured you didn't want to add an additional mime check just for some lynxchan fuckup.
Take this thread for example: https://nocsp.kohlchan.net/prog/res/1564.html

The problem is, the board software sometimes does not add the proper extension, so it will show up as ...imagepng instead of ...imagepng.png. DC only checks for the extension after the dot, so it won't be recognized. But lynxchan sends a data tag: document.getElementsByClassName("imgLink")[0].getAttribute("data-filemime").

SthephanShinkufag commented 5 years ago

This is a kohlchan software bug, not Dollchan.

StephenLynx commented 5 years ago

Is not a bug, old files don't have extensions.

SthephanShinkufag commented 5 years ago

What does mean "old files"? In old-api-lynxchan boards everything the same, links have extensions in href.

<a class="imgLink" target="_blank" href="https://endchan.xyz/.media/229c397141dfbf7eb7422d6e1f840ec8-imagejpeg.jpg" data-filewidth="558" data-fileheight="600" data-filemime="image/jpeg">
    <img src="/.media/t_229c397141dfbf7eb7422d6e1f840ec8-imagejpeg" title="[Click] открыть по центру, [Ctrl+Click] в посте" width="237" height="255">
</a>

StephenLynx commented 5 years ago

Old files are pre 1.7 files. Most sites are not that old, but sites that migrated their content, migrated to 1.5 and then upgraded the data.

ganego commented 5 years ago

And when was that? Because I see files in new threads from time to time that are bugged.
Is this because the file was not re-uploaded in this recent post but since it had the same MD5 as the same file uploaded sometime earlier, the old file was used?
So basically we need a shell script that reads the magicbytes of all media files on the server and sets the extension accordingly to fix this and change the image file names in the database. Until then, it's bugged.

Why would you even think in the first place, that saving files without extension makes any sense? But then you send a data-mimetype along. And how do you get this? Read the magicbytes each time, or waste database space by saving this information along?

StephenLynx commented 5 years ago

Is this because the file was not re-uploaded in this recent post but since it had the same MD5 as the same file uploaded sometime earlier, the old file was used?

Correct.

Until then, it's bugged.

Everything is working as intended, is not bugged.

Why would you even think in the first place, that saving files without extension makes any sense?

Extensions are not a standard, mimes are. I only added them back on 1.7 because too many people save files directly without using the link to download with the original filename, and desktops expect files to have extensions.

And how do you get this?

I use the mime the client tells me. Which is as valid as the extension the client tells me. Then yes, I store this information, which is negligible when compared to the actual space used by the file itself.

kr4ssi commented 5 years ago

https://github.com/SthephanShinkufag/Dollchan-Extension-Tools/pull/1344

SthephanShinkufag commented 5 years ago

@kr4ssi

Show me a link to the long-live post with that kind of video. I must test it.

kr4ssi commented 5 years ago

I haven't seen it happening to videos too yet, for images it works

ganego commented 5 years ago

There is a bug with post dates.
1) Open any thread in tab/browser A. 2) Write some post. 3) Open the same thread in a different tab/browser B.
-> Times should be correct. 4) In tab/browser B write a post.
5) Refresh A -> Time is incorrect.

And while at it: Deleting posts does not work.

ganego commented 5 years ago

Admin set css for video max-height: 90vh in https://kohlchan.net/.static/css/posting.css This will also affect dollchan making some videos not have full height (especially when min-webm-width is a bigger value).

Dollchan has to set video { max-height: 100%; }

kr4ssi commented 5 years ago

*.webp doesn't show up in preview.

ritz127 commented 5 years ago

kohlchan has a code in https://kohlchan.net/.static/js/posting.js which adds unix timestamp download links for each file, currently it's not working with dollchan. initial load adds all links correctly but this is ignore by dollchan's auto-updater

SthephanShinkufag commented 5 years ago

@ritz127 I don't understand what do you mean. I can't see any timestamp link. Show me a screenshot.

SthephanShinkufag commented 5 years ago

@StephenLynx

Please help. I recently implemented generating md5 for images and posting with fileMd5, fileMime and fileName fields, with checking through checkFileIdentifier.js for existed md5 entries. Also it is possible to post spoiler pictures. So, if md5 is existed then I send fileMd5, fileMime and fileName but don't send a file content. Otherwise I send fileMd5, fileMime and fileName and a content for the file. But it doesn't work in second case! If a md5 already existed then everything works, but if it does not existed on server - that image fails. For example I want to post 3 images, first and third are already existed on server and the second is new, After posting I will have only first and third in my post. The second image will be ignored.

Dollchan request parameters fields are seems NO DIFFERENT from parameters without a Dollchan. But with Dollchan I have a problem! I’ve been trying to find a problem for several days, but still without result.

Here the link to test script.. https://github.com/SthephanShinkufag/Dollchan-Extension-Tools/raw/26a560815863d72cb260e6881bec342163a00123/src/Dollchan_Extension_Tools.es6.user.js

How to check:

Try to post a new image with Dollchan. Posting doesn't work.
Post it without Dollchan. Posting works.
Post it again with Dollchan. Posting works!

(!) If you post on lynxhub.com/test then post some text without Dollchan first, to pass captcha. (!) Don't use Greasemonkey if you want to analyse for requuest parameters because network monitor in Firefox doesn't see Greasemonkey requests. (!) This is a test script, don't use it for regular posting on boards.

StephenLynx commented 5 years ago

Where are you appending the file content?

Here? if(isAppend) { data.append(name, val); }

If this is it, then you are missing the file name. Did you see how my code works? On api.js?

SthephanShinkufag commented 5 years ago

The problem is something else.

SthephanShinkufag commented 5 years ago

Also without

            data.append('fileMd5', md5Value);
            data.append('fileMime', mime);
            data.append('fileName', newFileName);

(as in stable script) Posting of new images works.

SthephanShinkufag commented 5 years ago

Without additional fields - posting of new images works.
With additional fields and without content - posting of existed images by md5 works.
With additional fields and with content - posting of new images does not work.

StephenLynx commented 5 years ago

1: the engine is just using no-js upload 2: the engine is just referencing images based on the md5+mime 3: something is mismatched between the actual file and the metadata (md5, mime, name)

StephenLynx commented 5 years ago

Btw, are you sending the spoiler status for all of them?

SthephanShinkufag commented 5 years ago

Yes, fileSpoiler for every attached file. As I said, I compared requests parameters in the network monitor of my browser, and everything seemed to be the same.

SthephanShinkufag commented 5 years ago

fileSpoiler parameter is sending through the checkbox element in the postform, that created by Dollchan.

StephenLynx commented 5 years ago

No, is not that. I am talking about individual file spoiler information. Again. DID YOU READ MY API.JS CODE?

SthephanShinkufag / Dollchan-Extension-Tools

[kohlchan.net]: Support discussion #1332