Closed schmittlauch closed 4 years ago
IBBoard
commented
4 years ago Huh, it's been a while since I've seen a "failed to allocate 2^63 bytes" error in Cawbird/Corebird.
Can you run a debug build (not just debug symbols) and see which of your mentions is the last one that it tries to parse? It's definitely doing something funky in the text transform. It's probably a corner-case value for something, and that code is all pure C so it's prone to memory handling issues.
schmittlauch
commented
4 years ago @IBBoard What needs to be taken care of for a full debug build, what flags need to be enabled? I'm not familiar with the vala ecosystem.
IBBoard
commented
4 years ago A basic meson build && ninja -C build should do it. It's all my local dev builds do. The default seems to be --buildtype=debug and the packaged builds explicitly supplies --buildtype=plain.
That will print the full JSON of every single message it receives. The last one it prints should be the one that it has problems with.
In future, I'll add some better logging so that we can see enough for issues like this from G_MESSAGES_DEBUG=cawbird cawbird
schmittlauch
commented
4 years ago This is the last message before the crash:
(cawbird:1644): dconf-WARNING **: 13:32:24.367: failed to commit changes to dconf: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name ca.desrt.dconf was not provided by any .service files
Message with type 12 on stream @schmittlauch
{
"created_at" : "Tue Dec 31 00:20:42 +0000 2019",
"id" : 1211804333978734592,
"id_str" : "1211804333978734592",
"full_text" : "@FlizzieMcGuire @schmittlauch https://t.co/30kMXiKMRU https://t.co/4Xxq6jHtm0",
"truncated" : false,
"display_text_range" : [
30,
53
],
"entities" : {
"hashtags" : [
],
"symbols" : [
],
"user_mentions" : [
{
"screen_name" : "FlizzieMcGuire",
"name" : "Bikini Bottom Mafia Stan Account",
"id" : 863773010,
"id_str" : "863773010",
"indices" : [
0,
15
]
},
{
"screen_name" : "schmittlauch",
"name" : "Trolli @schmittlauch@toot.matereal.eu 🦥",
"id" : 312869558,
"id_str" : "312869558",
"indices" : [
16,
29
]
}
],
"urls" : [
{
"url" : "https://t.co/30kMXiKMRU",
"expanded_url" : "https://twitter.com/chadloder/status/1211804049240031232?s=21",
"display_url" : "twitter.com/chadloder/stat…",
"indices" : [
54,
77
]
},
{
"url" : "https://t.co/4Xxq6jHtm0",
"expanded_url" : "https://twitter.com/chadloder/status/1211804049240031232",
"display_url" : "twitter.com/chadloder/stat…",
"indices" : [
54,
77
]
}
]
},
"source" : "<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>",
"in_reply_to_status_id" : 1211803346975240192,
"in_reply_to_status_id_str" : "1211803346975240192",
"in_reply_to_user_id" : 863773010,
"in_reply_to_user_id_str" : "863773010",
"in_reply_to_screen_name" : "FlizzieMcGuire",
"user" : {
"id" : 98575337,
"id_str" : "98575337",
"name" : "Chad Loder",
"screen_name" : "chadloder",
"location" : "Los Angeles, CA",
"description" : "Founder @Habitu8 • Recovering tech guy, author, investor • Human • Previously: Founder, VP Engineering @Rapid7 • #blacklivesmatter",
"url" : "https://t.co/j2ABO3HoJN",
"entities" : {
"url" : {
"urls" : [
{
"url" : "https://t.co/j2ABO3HoJN",
"expanded_url" : "https://www.habitu8.io/",
"display_url" : "habitu8.io",
"indices" : [
0,
23
]
}
]
},
"description" : {
"urls" : [
]
}
},
"protected" : false,
"followers_count" : 41256,
"friends_count" : 3984,
"listed_count" : 392,
"created_at" : "Tue Dec 22 07:11:56 +0000 2009",
"favourites_count" : 39913,
"utc_offset" : null,
"time_zone" : null,
"geo_enabled" : true,
"verified" : false,
"statuses_count" : 13848,
"lang" : null,
"contributors_enabled" : false,
"is_translator" : false,
"is_translation_enabled" : false,
"profile_background_color" : "C0DEED",
"profile_background_image_url" : "http://abs.twimg.com/images/themes/theme1/bg.png",
"profile_background_image_url_https" : "https://abs.twimg.com/images/themes/theme1/bg.png",
"profile_background_tile" : false,
"profile_image_url" : "http://pbs.twimg.com/profile_images/1213949167870984193/SohzlEa0_normal.jpg",
"profile_image_url_https" : "https://pbs.twimg.com/profile_images/1213949167870984193/SohzlEa0_normal.jpg",
"profile_banner_url" : "https://pbs.twimg.com/profile_banners/98575337/1578219983",
"profile_link_color" : "F0CBCA",
"profile_sidebar_border_color" : "C0DEED",
"profile_sidebar_fill_color" : "DDEEF6",
"profile_text_color" : "333333",
"profile_use_background_image" : true,
"has_extended_profile" : true,
"default_profile" : false,
"default_profile_image" : false,
"can_media_tag" : true,
"followed_by" : true,
"following" : false,
"follow_request_sent" : false,
"notifications" : false,
"translator_type" : "none"
},
"geo" : null,
"coordinates" : null,
"place" : null,
"contributors" : null,
"is_quote_status" : true,
"quoted_status_id" : 1211804049240031232,
"quoted_status_id_str" : "1211804049240031232",
"quoted_status_permalink" : {
"url" : "https://t.co/4Xxq6jHtm0",
"expanded" : "https://twitter.com/chadloder/status/1211804049240031232",
"display" : "twitter.com/chadloder/stat…"
},
"quoted_status" : {
"created_at" : "Tue Dec 31 00:19:34 +0000 2019",
"id" : 1211804049240031232,
"id_str" : "1211804049240031232",
"full_text" : "The Kiwifarms shit-stains are mad that Nazi-loving backpfeifengesicht¹ Vincent Canfield got booted from #36C3 conference.\n\nLet’s be clear.\n\n1. The hacking scene has ALWAYS had antifascists.\n\n2. Anti-antifascist literally means “fascist”.\n\n¹ - loosely translated, “punchable face” https://t.co/2wXVVs9An8",
"truncated" : false,
"display_text_range" : [
0,
279
],
"entities" : {
"hashtags" : [
{
"text" : "36C3",
"indices" : [
104,
109
]
}
],
"symbols" : [
],
"user_mentions" : [
],
"urls" : [
],
"media" : [
{
"id" : 1211804033226170368,
"id_str" : "1211804033226170368",
"indices" : [
280,
303
],
"media_url" : "http://pbs.twimg.com/media/ENExWQnU4AAcv5K.jpg",
"media_url_https" : "https://pbs.twimg.com/media/ENExWQnU4AAcv5K.jpg",
"url" : "https://t.co/2wXVVs9An8",
"display_url" : "pic.twitter.com/2wXVVs9An8",
"expanded_url" : "https://twitter.com/chadloder/status/1211804049240031232/photo/1",
"type" : "photo",
"sizes" : {
"thumb" : {
"w" : 150,
"h" : 150,
"resize" : "crop"
},
"medium" : {
"w" : 1024,
"h" : 496,
"resize" : "fit"
},
"large" : {
"w" : 1024,
"h" : 496,
"resize" : "fit"
},
"small" : {
"w" : 680,
"h" : 329,
"resize" : "fit"
}
},
"features" : {
"orig" : {
"faces" : [
]
},
"medium" : {
"faces" : [
]
},
"large" : {
"faces" : [
]
},
"small" : {
"faces" : [
]
}
}
}
]
},
"extended_entities" : {
"media" : [
{
"id" : 1211804033226170368,
"id_str" : "1211804033226170368",
"indices" : [
280,
303
],
"media_url" : "http://pbs.twimg.com/media/ENExWQnU4AAcv5K.jpg",
"media_url_https" : "https://pbs.twimg.com/media/ENExWQnU4AAcv5K.jpg",
"url" : "https://t.co/2wXVVs9An8",
"display_url" : "pic.twitter.com/2wXVVs9An8",
"expanded_url" : "https://twitter.com/chadloder/status/1211804049240031232/photo/1",
"type" : "photo",
"sizes" : {
"thumb" : {
"w" : 150,
"h" : 150,
"resize" : "crop"
},
"medium" : {
"w" : 1024,
"h" : 496,
"resize" : "fit"
},
"large" : {
"w" : 1024,
"h" : 496,
"resize" : "fit"
},
"small" : {
"w" : 680,
"h" : 329,
"resize" : "fit"
}
},
"features" : {
"orig" : {
"faces" : [
]
},
"medium" : {
"faces" : [
]
},
"large" : {
"faces" : [
]
},
"small" : {
"faces" : [
]
}
}
}
]
},
"source" : "<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>",
"in_reply_to_status_id" : null,
"in_reply_to_status_id_str" : null,
"in_reply_to_user_id" : null,
"in_reply_to_user_id_str" : null,
"in_reply_to_screen_name" : null,
"user" : {
"id" : 98575337,
"id_str" : "98575337",
"name" : "Chad Loder",
"screen_name" : "chadloder",
"location" : "Los Angeles, CA",
"description" : "Founder @Habitu8 • Recovering tech guy, author, investor • Human • Previously: Founder, VP Engineering @Rapid7 • #blacklivesmatter",
"url" : "https://t.co/j2ABO3HoJN",
"entities" : {
"url" : {
"urls" : [
{
"url" : "https://t.co/j2ABO3HoJN",
"expanded_url" : "https://www.habitu8.io/",
"display_url" : "habitu8.io",
"indices" : [
0,
23
]
}
]
},
"description" : {
"urls" : [
]
}
},
"protected" : false,
"followers_count" : 41256,
"friends_count" : 3984,
"listed_count" : 392,
"created_at" : "Tue Dec 22 07:11:56 +0000 2009",
"favourites_count" : 39913,
"utc_offset" : null,
"time_zone" : null,
"geo_enabled" : true,
"verified" : false,
"statuses_count" : 13848,
"lang" : null,
"contributors_enabled" : false,
"is_translator" : false,
"is_translation_enabled" : false,
"profile_background_color" : "C0DEED",
"profile_background_image_url" : "http://abs.twimg.com/images/themes/theme1/bg.png",
"profile_background_image_url_https" : "https://abs.twimg.com/images/themes/theme1/bg.png",
"profile_background_tile" : false,
"profile_image_url" : "http://pbs.twimg.com/profile_images/1213949167870984193/SohzlEa0_normal.jpg",
"profile_image_url_https" : "https://pbs.twimg.com/profile_images/1213949167870984193/SohzlEa0_normal.jpg",
"profile_banner_url" : "https://pbs.twimg.com/profile_banners/98575337/1578219983",
"profile_link_color" : "F0CBCA",
"profile_sidebar_border_color" : "C0DEED",
"profile_sidebar_fill_color" : "DDEEF6",
"profile_text_color" : "333333",
"profile_use_background_image" : true,
"has_extended_profile" : true,
"default_profile" : false,
"default_profile_image" : false,
"can_media_tag" : true,
"followed_by" : true,
"following" : false,
"follow_request_sent" : false,
"notifications" : false,
"translator_type" : "none"
},
"geo" : null,
"coordinates" : null,
"place" : null,
"contributors" : null,
"is_quote_status" : false,
"retweet_count" : 4,
"favorite_count" : 18,
"favorited" : false,
"retweeted" : false,
"possibly_sensitive" : false,
"lang" : "en"
},
"retweet_count" : 0,
"favorite_count" : 1,
"favorited" : false,
"retweeted" : false,
"possibly_sensitive" : false,
"lang" : "und"
}
(cawbird:1644): GLib-ERROR **: 13:32:24.368: ../glib/gmem.c:105: failed to allocate 18446744073709551594 bytesHope that's helpful. I also have a bunch of filter regexes active, but I don't know where to find them.
IBBoard
commented
4 years ago That should help. There's only three substring calls in the parsing function, so I'm hoping it's not too hard to track down when I get a chance. Unfortunately, I spent this evening fighting other tests that I hadn't realised were failing so that I could get this tested properly.
Filters should be visible under the funnel symbol - it'll show blocked accounts, and above that it'll show regex filters.
schmittlauch
commented
4 years ago Filters should be visible under the funnel symbol - it'll show blocked accounts, and above that it'll show regex filters.
Reminder that Cawbird crashes directly at launch - so there's no time for clicking around the UI ;) I'd be more interested in the location of the files it stores them in.
IBBoard
commented
4 years ago D'oh, of course! Sorry.
~/.config/cawbird/accounts/<account_id>.db - it's an SQLite database with a table called "filters" in it.
The stack trace mentions filters, but from a quick skim of the code then I didn't think the filters had been applied yet. I'll confirm tonight and try to fix.
IBBoard
commented
4 years ago Debugging this now. Got a failing test. Found that it's caused by trying to do a substring from a position to an earlier position (i.e. negative length). Tried to find what calculation went wrong.
Looks like it's Twitter feeding us bad data!
"urls" : [
{
"url" : "https://t.co/30kMXiKMRU",
"expanded_url" : "https://twitter.com/chadloder/status/1211804049240031232?s=21",
"display_url" : "twitter.com/chadloder/stat…",
"indices" : [
54,
77
]
},
{
"url" : "https://t.co/4Xxq6jHtm0",
"expanded_url" : "https://twitter.com/chadloder/status/1211804049240031232",
"display_url" : "twitter.com/chadloder/stat…",
"indices" : [
54,
77
]
}
]Notice how the two URLs are different but their indices are the same. The expanded URLs are to the same tweet, but one has a tracking query string.
Now to work out how if we can fix this, because we're missing important data about where entities are!
IBBoard
commented
4 years ago Okay, it's not perfect (one link would be un-linkified), but we've got a "fix" in for it now. It looks like the original tweet has gone, though, so I can't test this in the app, but the unit test should have us covered.
schmittlauch
commented
4 years ago I just tried building it, it fails due to the new text failing. Isn't it the case that a failure of what you're testing for should actually make the test pass?
Compilation succeeded - 5 warning(s)
[153/179] Compiling Vala source /build/source/vapi/cawbird-internal.vapi /build/source/vapi/rest-0.7.vapi ../tests/texttransform.vala cawbird.vapi.vapi.
FAILED: tests/59830eb@@texttransform@exe/texttransform.c
valac -C --enable-deprecated -D GTK322 -D VIDEO -D SPELLCHECK --pkg gspell-1 --pkg gstreamer-video-1.0 --pkg libsoup-2.4 --pkg sqlite3 --pkg json-glib-1.0 --pkg gtk+-3.0 --target-glib 2.44 --pkg glib-2.0 --color=always --directory tests/59830eb@@texttransform@exe --basedir ../tests --gresources=../cawbird.gresource.xml --gresources=/build/source/cawbird.gresource.xml /build/source/vapi/cawbird-internal.vapi /build/source/vapi/rest-0.7.vapi ../tests/texttransform.vala cawbird.vapi
../tests/texttransform.vala:21.5-21.25: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "bar",
^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:45.5-45.25: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "foo",
^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:70.5-70.26: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#foo",
^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:90.5-90.26: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#foo",
^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:112.5-112.44: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "http://t.co/O5uZwJg31k",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:120.5-120.44: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "http://t.co/BsKkxv8UG4",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:128.5-128.44: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "http://t.co/W8qs846ude",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:136.5-136.44: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "http://t.co/x4bKoCusvQ",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:163.5-163.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#totally",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:171.5-171.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "@baedert",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:179.5-179.37: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#baedertworship",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:187.5-187.31: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#thefeels",
^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:211.5-211.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#totally",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:219.5-219.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "@baedert",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:227.5-227.37: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#baedertworship",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:235.5-235.31: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#thefeels",
^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:243.5-243.29: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#foobar",
^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:268.5-268.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#totally",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:276.5-276.37: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#baedertworship",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:284.5-284.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "@baedert",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:292.5-292.29: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#foobar",
^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:316.5-316.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#totally",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:324.5-324.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "@baedert",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:332.5-332.37: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#baedertworship",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:340.5-340.31: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#thefeels",
^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:348.5-348.29: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#foobar",
^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:373.5-373.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#totally",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:381.5-381.30: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "@baedert",
^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:389.5-389.37: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "#baedertworship",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:397.5-397.40: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "https://foobar.com",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../tests/texttransform.vala:425.5-425.28: error: Invalid member `original_text' in `Cb.TextEntity'
original_text = "Foobar",
^^^^^^^^^^^^^^^^^^^^^^^^
Compilation failed: 31 error(s), 0 warning(s)
ninja: build stopped: subcommand failed.
builder for '/nix/store/3shv0808f66jhdlq8hkdl0gl54shjkj7-cawbird-1.0.3.1.drv' failed with exit code 1
error: build of '/nix/store/3shv0808f66jhdlq8hkdl0gl54shjkj7-cawbird-1.0.3.1.drv' failedNevermind, I accidentally build from badb69b038ad570a06f2891dd3896fec0b4bf5b0 instead of d8990dadff90ca22d856f183628b30480cf99c30.
Now it works again, although I get interesting warnings like
(cawbird:25568): cawbird-WARNING **: 22:13:39.792: ../src/CbUtils.c(call_done_cb): 0x21558a0, Not Found
(cawbird:25568): cawbird-WARNING **: 22:13:39.792: Twitter.vala:233: Not Found
(cawbird:25568): cawbird-WARNING **: 22:13:41.577: ../src/CbUtils.c(call_done_cb): 0x217f160, Not Found
(cawbird:25568): cawbird-WARNING **: 22:13:41.577: Twitter.vala:233: Not FoundBut your workarounds are active indeed:
cawbird-INFO: 22:16:44.283: Skipping entity - expected https://t.co/30kMXiKMRU but found https://t.co/4Xxq6jHtm0. Likely bad indices (54 to 77)
(cawbird:32082): dconf-WARNING **: 22:16:44.290: failed to commit changes to dconf: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name ca.desrt.dconf was not provided by any .service files
cawbird-INFO: 22:16:44.291: Skipping entity - expected https://t.co/30kMXiKMRU but found https://t.co/4Xxq6jHtm0. Likely bad indices (54 to 77)What are these indices?
And how do you want to proceed? Release this workaround as a minor update or continue working on it?
No matter what, thank you for your efforts (:
IBBoard
commented
4 years ago Nevermind, I accidentally build from badb69b instead of d8990da.
1) I did think that the GitHub UI wasn't too clear that there was a second commit! 2) Ooops, I thought I'd made "clean" commits that would build independently. I forgot that I had to change lots of code in the tests to add the new field. My bad.
Now it works again, although I get interesting warnings like
Twitter.vala line 233 is it trying to load the JSON for a person and failing. Presumably a user account doesn't exist any more. Either that or I managed to unintentionally break something.
But your workarounds are active indeed:
Do you still have that tweet in your mentions? Because I tried to open it (based on the ID and sender) and got Not Found.
What are these indices?
They're the positions (indexes) into the string that the entities (hashtags, @-handles or URLs) are at. It's Twitter's way of giving you a 140 character text string plus some objects to turn that into a hyperlinked message.
And how do you want to proceed? Release this workaround as a minor update or continue working on it?
I'll make a release later in the week. I don't think there's anything better that we can do. There's lost data, so "skip the malformed entity and leave it as text" is the best we can do. It'd be nice if it was a warning instead of an INFO, but I can live with that.
The good news is that someone at Twitter is intrigued by the bad data and wants to investigate!
schmittlauch
commented
4 years ago @IBBoard this one looks a bit suspicious: https://twitter.com/chadloder/status/1211804333978734592
Edit: This seems to be a true positive:
cawbird-INFO: 23:45:41.703: Skipping entity - expected https://t.co/30kMXiKMRU but found https://t.co/4Xxq6jHtm0. Likely bad indices (54 to 77)
cawbird-INFO: 23:45:41.894: Skipping entity - expected https://t.co/30kMXiKMRU but found https://t.co/4Xxq6jHtm0. Likely bad indices (54 to 77)Your workaround is a bit too cautious, as it also gets triggered when the case of a Twitter nick doesn't match:
cawbird-INFO: 23:47:15.522: Skipping entity - expected @PerspektiveOn but found @perspektiveon. Likely bad indices (157 to 171)
https://twitter.com/KarieMuster/status/1214669365687263233
I suggest normalising the case before comparison. Hashtags could be affected as well.
schmittlauch
commented
4 years ago another false positive with hashtags in the profile description/ bio:
cawbird-INFO: 23:11:42.197: Skipping entity - expected (null) but found #Demokratie. Likely bad indices (35 to 46)
cawbird-INFO: 23:11:42.198: Skipping entity - expected (null) but found #Toleranz. Likely bad indices (51 to 60)
cawbird-INFO: 23:11:42.198: Skipping entity - expected (null) but found #Bautzen. Likely bad indices (74 to 82)for https://twitter.com/Schmanle
Also happens with @mentions in the bio: https://twitter.com/patrizipien And links: https://twitter.com/MInkorrekt
IBBoard
commented
4 years ago @IBBoard this one looks a bit suspicious: https://twitter.com/chadloder/status/1211804333978734592
Edit: This seems to be a true positive:
On the plus side, at least Twitter Web breaks on it as well, so I'm doing no worse than they are!
Although damnit, I just sent that link to the Twitter dev and insta-crashed my unpatched Cawbird 😆
Your workaround is a bit too cautious, as it also gets triggered when the case of a Twitter nick doesn't match:
Ah, I hadn't thought about case sensitivity. I'd assumed that case would match (because it was telling you which text it was highlighting). Should be easy enough to fix.
another false positive with hashtags in the profile description/ bio:
D'oh! That's going to be a bit more annoying. I'll have to look at that and see what data it has.
Thanks for the extra testing.
IBBoard
commented
4 years ago Can you try 93bbcb8 and see how it goes?
schmittlauch
commented
4 years ago Looks good so far. It doesn't even choke on the #Über I put into my bio or testing purposes.
schmittlauch
commented
4 years ago Here comes unicode normalisation:
cawbird-INFO: 01:05:29.524: Skipping entity - expected #プロスピ but found ＃プロスピ. Likely bad indices (25 to 30)
cawbird-INFO: 01:05:29.524: Skipping entity - expected #リーグオーダー but found ＃リーグオーダー. Likely bad indices (33 to 41)Gah! Curse Unicode!
At first I thought they were the same, but then I ran them through hexdump and I can see that one uses a U+FF03 FULLWIDTH NUMBER SIGN to go with the full-width Katakana characters.
I hadn't accounted for the hash being anything other than #. It seems the case-insensitivity still works. I just need a fix for the hash symbol.
IBBoard
commented
4 years ago That turned out to be a simple problem with a solution made far more complex by C and not being able to take shortcuts with Unicode characters!
schmittlauch
commented
4 years ago Looks good. I browsed through a bunch of Asian hashtags and never got an INFO message about skipping.
IBBoard
commented
4 years ago I've not seen any more issues. I'm assuming this is fixed now.
Cawbird reproducably crashes at each start and dumps core with
(cawbird:16027): GLib-ERROR **: 16:27:51.918: ../glib/gmem.c:105: failed to allocate 18446744073709551594 bytes.This is the backtrace from the coredump file: Terminal output:
If it is necessary to reproduce this with debug symbols enabled, I can try to do so.
Platform: NixOS 19.09 Version: 1.0.3.1 library versions are visible in the backtrace paths