Chasing dead links

[tlienart]

Right so I ran a script to check all pages, it's not perfect but it should help us find quite a few of the deadlinks, list below. Maybe that people could take responsibility in fixing things and pinging me in the PR so that I can mark it as done here.

There's a bunch of dead meetups etc that should probably just be removed

@ Helpers: there's two things you can do:

• do you recognise a link? do you know a fix? --> open a PR with the fix
• page is dead? just remove the link

Edit:

• fixed a bunch of issues due to use of https instead of http
• removed from the list github links that errorred with 429 (too many requests), some may still be broken but most are fine
• removed 999 (linkedin)
• pruned from 600 to 100, yay!

Script

(skip this, it's only for maintenance purpose, I might eventually put this in the README)

It uses blc and is not pretty but it kind of does an ok job:

# modify this to wherever your npm installs stuff
const BLC = "/usr/local/Cellar/node/11.10.1/lib/node_modules/broken-link-checker/bin/blc"
@assert success(`$BLC -V`)
function check_page(url)
    open("tempf", "w") do outf
        redirect_stdout(outf) do
            try run(`$BLC $url`); catch; end # sometimes BLC does weird stuff
        end
    end
    output = readlines("tempf")
    rm("tempf")
    for line in output
        startswith(line, "├─BROKEN─") || continue
        tmp = replace(line, "├─BROKEN─ " => "")
        println("  * [ ] $tmp")
    end
end
# modify this to your local version of the site, assumes you've built it.
const BASE_PATH = "/Users/tlienart/Desktop/www.julialang.org/__site"
for (root, _, files) in walkdir(BASE_PATH)
    for file in files
        file == "index.html" || continue
        fp = replace(joinpath(root, file), BASE_PATH => "")
        fp = "https://julialang.org" * fp
        println("* [ ] $fp")
        check_page(fp)
    end
end

List of dead links

Notes:

• the BLC error codes are indicated, 404 is the most obvious one; there are some others which are due to the robot being kicked so we might have to do this manually.
• BLC_UNKNOWN are usually related to a https instead of http

Errors

Chunk 1 (checked)

• [ ] https://julialang.org/blog/2012/02/why-we-created-julia/index.html
  • [ ] https://topsy.com/julialang.org/ (ERRNO_ENOTFOUND)
  • [ ] https://sd.csdn.net/a/20120223/312315.html (ERRNO_ENOTFOUND)
  • [ ] https://www.statalgo.com/2012/03/24/statistics-with-julia/ (HTTP 404)
• [ ] https://julialang.org/blog/2012/02/why-we-created-julia-zh_CN/index.html
  • [ ] https://topsy.com/julialang.org/ (ERRNO_ENOTFOUND)
  • [ ] https://sd.csdn.net/a/20120223/312315.html (ERRNO_ENOTFOUND)
  • [ ] https://timsalimans.com/gibbs-sampling-with-julia/ (ERRNO EPROTO)
  • [ ] https://www.statalgo.com/2012/03/24/statistics-with-julia/ (HTTP 404)
• [ ] https://julialang.org/blog/2013/03/julia-tutorial-MIT/index.html
  • [ ] https://docs.julialang.org/en/release-0.1/stdlib/base/#signal_processing (HTTP 404)
  • [ ] https://github.com/JuliaStats/LM.jl (HTTP 404)

Chunk 2 (checked)

• [ ] https://julialang.org/blog/2013/04/distributed-numerical-optimization/index.html
  • [ ] https://www.springerreference.com/docs/html/chapterdbid/72429.html (BLC UNKNOWN)

Chunk 4

• [ ] https://julialang.org/blog/2016/03/parallelaccelerator/index.html
  • [ ] https://docs.julialang.org/en/release-0.4/manual/arrays/#comprehensions (HTTP 404)
  • [ ] https://docs.julialang.org/en/release-0.4/manual/arrays/#comprehensions (HTTP 404)
  • [ ] https://www.cs.indiana.edu/~lkuper/ (HTTP 404)
  • [ ] https://docs.julialang.org/en/release-0.4/manual/conversion-and-promotion/#conversion (HTTP 404)
  • [ ] https://docs.julialang.org/en/release-0.4/manual/functions/#do-block-syntax-for-function-arguments (HTTP 404)
  • [ ] https://docs.julialang.org/en/release-0.4/devdocs/reflection/ (HTTP 404)
• [ ] https://julialang.org/blog/2016/05/juliacon-invitation/index.html
  • [ ] https://juliacon.org/workshops.html (HTTP 404)
  • [ ] https://juliacon.org/abstracts.html (HTTP 404)
  • [ ] https://juliacon.org/pdf/juliacon2016poster3.pdf (HTTP 404)
  • [ ] https://juliacon.org/pdf/juliacon2016poster3.pdf (HTTP 404)

Chunk 6

• [ ] https://julialang.org/blog/2016/10/julia-0.5-highlights/index.html
  • [ ] https://docs.julialang.org/en/release-0.5/stdlib/math/#Base.clamp (HTTP 404)
  • [ ] https://julialang.org/blog/2016/02/multidimensional-algorithms-and-iteration/ (HTTP 404)
  • [ ] https://julialang.org/blog/2016/03/generalizing-abstractarrays-opportunities-and-challenges/ (HTTP 404)
• [ ] https://julialang.org/blog/2017/01/moredots/index.html
  • [ ] https://docs.julialang.org/en/latest/manual/strings.html#Regular-Expressions-1 (HTTP 404)

Chunk 8

• [ ] https://julialang.org/blog/2017/04/repl-0.6-highlights/index.html
  • [ ] https://docs.julialang.org/en/stable/manual/interacting-with-julia/ (HTTP 404)
• [ ] https://julialang.org/blog/2017/08/juliacon/index.html
  • [ ] https://www.youtube.com/watch?v=7NDkpWoNiQ4 (HTTP 404)
  • [ ] https://www.youtube.com/watch?v=DUdE3M2nlDE (HTTP 404)
  • [ ] https://www.youtube.com/watch?v=FKBSVb9405w (HTTP 404)
  • [ ] https://www.youtube.com/watch?v=uMs2192YAxg (HTTP 404)
• [ ] https://julialang.org/blog/2017/09/gsoc-fenics/index.html
  • [ ] https://docs.julialang.org/en/latest/stdlib/arrays/ (HTTP 404)

Chunk 9

• [ ] https://julialang.org/blog/2017/12/ml&pl/index.html
  • [ ] https://mxnet.incubator.apache.org/api/python/gluon.html (HTTP 404)
  • [ ] https://mxnet.incubator.apache.org/how_to/multi_devices.html (HTTP 404)
  • [ ] https://mxnet.incubator.apache.org/api/python/gluon.html (HTTP 404)
  • [ ] https://github.com/google/sling/tree/master/myelin (HTTP 404)
  • [ ] https://www.tensorflow.org/api_docs/python/tf/TextLineReader (HTTP 404)
• [ ] https://julialang.org/blog/2017/12/ml&pl-cn/index.html
  • [ ] https://zh.wikipedia.org/zh-hk/兒童黑話 (BLC UNKNOWN)
  • [ ] https://zh.wikipedia.org/zh-hk/兒童黑話 (HTTP undefined)
  • [ ] https://mxnet.incubator.apache.org/api/python/gluon.html (HTTP 404)
  • [ ] https://mxnet.incubator.apache.org/how_to/multi_devices.html (HTTP 404)
  • [ ] https://mxnet.incubator.apache.org/api/python/gluon.html (HTTP 404)
  • [ ] https://github.com/google/sling/tree/master/myelin (HTTP 404)
  • [ ] https://www.tensorflow.org/api_docs/python/tf/TextLineReader (HTTP 404)
• [ ] https://julialang.org/blog/2017/12/ml&pl-zh_tw/index.html
  • [ ] https://mxnet.incubator.apache.org/api/python/gluon.html (HTTP 404)
  • [ ] https://mxnet.incubator.apache.org/how_to/multi_devices.html (HTTP 404)
  • [ ] https://mxnet.incubator.apache.org/api/python/gluon.html (HTTP 404)
  • [ ] https://github.com/google/sling/tree/master/myelin (HTTP 404)
  • [ ] https://www.tensorflow.org/api_docs/python/tf/TextLineReader (HTTP 404)
• [ ] https://julialang.org/blog/2018/06/missing/index.html
  • [ ] https://arrow.apache.org/docs/memory_layout.html#null-bitmaps (HTTP 404)
• [ ] https://julialang.org/blog/2018/07/iterators-in-julia-0.7/index.html
  • [ ] https://docs.julialang.org/en/release-0.6/manual/interfaces/#man-interface-iteration-1 (HTTP 404)

Chunk 10

• [ ] https://julialang.org/blog/2018/08/GSoC-Final-Summary/index.html
  • [ ] https://github.com/tejank10/Flux-baselines/blob/master/dqn/dqn.jl (HTTP 404)
  • [ ] https://github.com/tejank10/Flux-baselines/blob/master/dqn/double-dqn.jl (HTTP 404)
  • [ ] https://github.com/tejank10/Flux-baselines/blob/master/dqn/duel-dqn.jl (HTTP 404)
  • [ ] https://github.com/tejank10/Flux-baselines/blob/master/actor-critic/a2c.jl (HTTP 404)
  • [ ] https://github.com/tejank10/Flux-baselines/tree/master/ddpg (HTTP 404)
  • [ ] https://github.com/tejank10/Flux-baselines/blob/master/actor-critic/a2c.jl (HTTP 404)
  • [ ] https://github.com/tejank10/Flux-baselines/blob/master/dqn/duel-dqn.jl (HTTP 404)
  • [ ] https://github.com/tejank10/Flux-baselines/tree/master/ddpg (HTTP 404)
  • [ ] https://github.com/tejank10/Flux-baselines/blob/master/dqn/prioritized-replay-dqn.jl (HTTP 404)
  • [ ] https://github.com/tejank10/model-zoo/tree/DNI/vision/mnist/dni.jl (HTTP 404)
  • [ ] https://github.com/tejank10/model-zoo/tree/DNI/vision/mnist/dni.jl (HTTP 404)
• [ ] https://julialang.org/blog/2018/08/adding-newer-features-and-speeding-up-convolutions-in-flux/index.html
  • [ ] https://avik-pal.github.io/blog/2018/07/Computer_Vision_using_Metalhead/ (HTTP 404)
• [ ] https://julialang.org/blog/2018/08/one-point-zero-es/index.html
  • [ ] https://docs.julialang.org/en/release-0.7/NEWS/ (HTTP 404)
• [ ] https://julialang.org/blog/2019/01/fluxdiffeq/index.html
  • [ ] https://julialang.org/blog/2019/04/fluxdiffeq-zh_tw/]Traditional%20Chinese (HTTP 404)

Chunk 11

• [ ] https://julialang.org/blog/2019/09/julia-workshop-beijing/index.html
  • [ ] https://www.bilibili.com/video/av65371788 (HTTP_403)
• [ ] https://julialang.org/blog/2019/09/julia-workshop-beijing-zh_cn/index.html
  • [ ] https://www.bilibili.com/video/av65371788 (HTTP_403)
• [ ] https://julialang.org/blog/2019/09/profilers/index.html
  • [ ] https://docs.junolab.org/stable/man/juno_frontend.html#Profiler-1 (HTTP 404)
  • [ ] https://github.com/JuliaLang/julia/tree/jn/profile-recurflat (HTTP 404)
• [ ] https://julialang.org/community/index.html
  • [ ] https://www.meetup.com/Bangalore-Julia-User-Group/ (HTTP 404)
  • [ ] https://www.meetup.com/Juliadelhi/ (HTTP 404)
  • [ ] https://www.meetup.com/Singapore-Julia-User-Group/ (HTTP 404)
  • [ ] https://www.meetup.com/JuliaStanbul/ (HTTP 404)
  • [ ] https://www.meetup.com/Montreal-Julia-Programming-Language-Meetup/ (HTTP 404)
  • [ ] https://www.meetup.com/Machine-Deep-Learning-an-Introduction/ (HTTP 404)
  • [ ] https://www.meetup.com/Vancouver-Julia-Users/ (HTTP 404)
  • [ ] https://www.meetup.com/Julia-lang-bsb/ (HTTP 404)
• [ ] https://julialang.org/downloads/oldreleases/index.html
  • [ ] https://junolab.s3.amazonaws.com/release/1.0.3/juno-windows-x32.zip (HTTP_403)
  • [ ] https://junolab.s3.amazonaws.com/release/1.0.3/juno-windows-x64.zip (HTTP_403)
  • [ ] https://junolab.s3.amazonaws.com/release/1.0.3/juno-mac-x64.dmg (HTTP_403)
  • [ ] https://junolab.s3.amazonaws.com/release/1.0.3/juno-linux-x32.zip (HTTP_403)
  • [ ] https://junolab.s3.amazonaws.com/release/1.0.3/juno-linux-x64.zip (HTTP_403)

Chunk 14

• [ ] https://julialang.org/learning/index.html
  • [ ] https://syl1.gitbook.io/julia-language-a-concise-tutorial/ (HTTP_204)
  • [ ] https://homerreid.dyndns.org/teaching/18.330/ (ERRNO ECONNREFUSED)
  • [ ] https://www.packtpub.com/application-development/mastering-julia-10 (HTTP 404)
  • [ ] https://www.youtube.com/watch?v=0-Xsts_s5W (HTTP 404)

Chunk 15

• [ ] https://julialang.org/research/index.html
  • [ ] https://dx.doi.org/https:/doi.org/10.1016/j.softx.2019.03.005 (HTTP 404)
  • [ ] https://dx.doi.org/10.1515/gcc-2018-0004 (HTTP 404)

[tlienart]

I realise a bunch of these BLC_UNKNOWN errors are due to me adding https: instead of http: massively. E.g.:

http://www-math.mit.edu/~edelman/ vs https://www-math.mit.edu/~edelman/

[ViralBShah]

For that particular one, the right link is https://math.mit.edu/~edelman/

[tlienart]

oh man this is hard work...

[tlienart]

alright did about half, many of which were due to the HTTPS. Another low hanging fruit are all the github ones and error 429...

[ViralBShah]

Thank you for this tireless effort!

[ViralBShah]

The github URLs all appear valid. What's going on?

[ViralBShah]

We should announce on #website and #general to look for help, but perhaps remove the 429 and 999 URLs from this list?

[tlienart]

Yeah I think GitHub may have a strict robot, I’ll do another pass to do more pruning.

One thing that would be good is advice for dead pages, do we just remove the link or do we replace it with an indication that there was a link that’s now dead?

[ViralBShah]

Not sure if it is worth the effort to do more. I think we can just remove.

[tlienart]

no no, not done yet...

[ViralBShah]

Bump. Help appreciated here.

[tlienart]

I imagine a few of those are not relevant anymore, would be good to re-run the script (bottom of readme, I'm on my phone now so I can't do it but can try later)

[ashwani-rathee]

So all the errors are left to be worked on (after chunk 4),right??

[tlienart]

yes that's correct; this list might be a bit outdated now but basically the steps are:

1. is the faulty link still there?
   1. yes --> is there an alternative link that works?
   2. yes --> change it in a PR (ideally do multiple links per PRs though maybe at most 20-30 to facilitate reviewing)
   3. no --> remove the link
   4. no --> reply here with a bunch of links that are irrelevant; I'll update the list

that's about it; then we should re-run the tool to see if there are any stray links in the mix.

[HarshCasper]

Hi @tlienart @ViralBShah

I would like to work on this Issue back again. Since this issue has been open for a very long time now, and major chunks of dead links are still left to be reviewed and fixed, I would like to take it up and clean it to give the Project a better shape.

I have recently learned about Julia and would like to contribute to Julia Open-Source in all ways possible. Kindly let me know if I can start working on it, or is there any other obligation I need to fulfil for the same.

I am looking to get started with contributing to Julia with this 😄

[ViralBShah]

Thanks @HarshCasper. Just open PRs fixing the dead links. You can do so in batches. Don't mix any other changes into the PRs that are fixing the dead links - so that they can be quickly merged.

[ViralBShah]

I think we just have to leave the external links as they are. It's impossible to play catch up.