this is a work in progress, later move this to Wiki ...
open (bigger) issues / problems / questions
technical
define scope: what is it, should it only check links or also references etc.?
(check) external link checking does not scale: will result in excessive crawling of external sites, esp. if the sum of all linkvalidator / brofix installations are considered - how to deal with this?
(check) external link checking: false positives
long term maintenance: should brofix and linkvalidator be 2 separate extensions, this divides available resources, ideally should share common code
(ui) link report is sluggish because entire page tree is crawled on every load
(ui|check) link check results possibly not up to date, because
link target might have changed status
link source: link no longer in field or element is not visible or deleted
(ui) link report not well usable
(ui) multiple occurances of same link target must be edited (e.g. link targethttps://example.org/foobasel exists several times)
organizational, strategic
address problem of long review wait time and contribution - how to get more people on board?
how to coordinate development (linkvalidator)?
where are core ideas / vision / specification for new features published and discussed?
solutions / ideas for the open issues (and beyond)
for 2: external link checking:
ideas:
there are several possible solutions for this which can be combined:
URL cache and throtting (e.g. crawl delay, max number of URLs per domain per check, read robotx.txt)
get information about URL status from other sources (in addition or instead of automatic checking), such as manual, using info from proxy etc.
centralize link checking in a service application (see also Wordpress solutions, join forces with agency, TYPO3 GmbH or SEO company such as AHrefs or Semrush) - additional possibilities to monetize by providing a free service for low volume and paid service per volume. A broken link on the source site is a 404 on the target site!
Some of this is already implemented in brofix (URL cache, crawl delay), the exclude URL / domain button also contributes to reducing number of external URLs.
In linkvalidator, this is currently solved by deactivating external link checking by default (since v13).
Both brofix and linkvalidator point this problem out in the documentation.
for 3: false positives (external URLs):
ideas:
exclude button
show URL check result as "uncheckable", if URL cannot be checked (e.g. for 401, 403)
in combination with (1), find solutions and partner who has solution
already implemented:
brofix: exclude button is implemented
brofix: will detect some results codes (e.g. cloudflare, 401, 403) and mark link not as broken but as "cannot check"
for 4: long term maintenance brofix/linkvalidator:
ideas:
ideally not duplicate entire source from linkvalitor / brofix, have brofix extend linkvalidator or at least use some basic functionality from linkvalidator
linkvalidator could be split up into a library and an application
for 5: link report sluggish / page tree:
idea: open edit form as module dialog. Currently, we open list, then edit, open list again: each time, the list is reloaded. This is not necessary. The edit form can be opened as module dialog, on close, only the affected entries can be updated (via JavaScript).
for 7: link report not well usable:
is already improved in brofix with sort, filter, paginate
for 8: multiple fixing of same link target:
(both probably out of scope but can be explored as separate extension)
have a suggest wizard in link browser which shows latests URLs that were used (for same previous URL)
provide tools for changing links across entire site
_for 20: better coordination
Additional cool ideas
these are some ideas off top of my head, needs to be explored further, not all make sense
:smile:
filter page tree for broken links: This might be possible in combination with EXT:pagetreefilter, add a toggle button (on/off). If on, the page tree is filtered to show only pages with broken links. In addition an icon is displayed for the page which contains broken links
in addition to list of broken links, create list of link targets (URLs) and list of domains. The current "exclude list" can be integrated into this. Additional information and configuration can be done directly on a domain and on a URL, e.g.
this is a work in progress, later move this to Wiki ...
open (bigger) issues / problems / questions
technical
https://example.org/foobaselexists several times)organizational, strategic
solutions / ideas for the open issues (and beyond)
for 2: external link checking:
ideas:
there are several possible solutions for this which can be combined:
https://ahrefs.com/broken-link-checker
already implemented:
Some of this is already implemented in brofix (URL cache, crawl delay), the exclude URL / domain button also contributes to reducing number of external URLs.
In linkvalidator, this is currently solved by deactivating external link checking by default (since v13).
Both brofix and linkvalidator point this problem out in the documentation.
for 3: false positives (external URLs):
ideas:
already implemented:
brofix: exclude button is implemented
brofix: will detect some results codes (e.g. cloudflare, 401, 403) and mark link not as broken but as "cannot check"
for 4: long term maintenance brofix/linkvalidator:
ideas:
for 5: link report sluggish / page tree:
for 7: link report not well usable:
for 8: multiple fixing of same link target:
(both probably out of scope but can be explored as separate extension)
_for 20: better coordination
Additional cool ideas
these are some ideas off top of my head, needs to be explored further, not all make sense
:smile:
Additional resources