Validate pull requests with Travis

awesome-bot commented 8 years ago
Hello, I wrote a tool that can validate README links (valid URLs, not duplicate). It can be run when someone submits a pull request.
It is currently being used by
Examples
https://travis-ci.org/matteocrippa/awesome-swift/builds/96526196 ok ✅
https://travis-ci.org/matteocrippa/awesome-swift/builds/96722421 link redirected / rename 🔴
https://travis-ci.org/dkhamsing/open-source-ios-apps/builds/96763135 bad link / project deleted 🔴
https://travis-ci.org/dkhamsing/open-source-ios-apps/builds/95754715 dupe 🔴
If you are interested, connect this repo to https://travis-ci.org/ and add a .travis.yml file to the project.
See https://github.com/dkhamsing/awesome_bot for options, more information Feel free to leave a comment :smile:
nobodxbodon commented 11 months ago
I was wondering if such tool existed. Let's try it out, to save the manual checking like here.
nobodxbodon commented 10 months ago
Just ran the checking as below (there were ~90 redirects without --allow-redirect). Seems 7 broken urls among 167, better than I thought.
% awesome_bot README.md --allow-redirect
> Checking links in README.md
> Will allow redirects
Links to check: 172, 167 unique
  001. https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg 
  002. https://github.com/sindresorhus/awesome 
  003. http://blog.gittip.com/post/26350459746/the-first-open-company/ 
  004. http://gratipay.com 
  005. https://en.wikipedia.org/wiki/Free_and_open-source_software 
  006. https://en.wikipedia.org/wiki/Free_content 
  007. https://en.wikipedia.org/wiki/Open-source_hardware 
  008. https://en.wikipedia.org/wiki/Open_standard 
  009. https://en.wikipedia.org/wiki/Interoperability 
  010. https://en.wikipedia.org/wiki/Open_business 
  011. http://p2pfoundation.net/Open_Business 
  012. http://www.google.com/s2/favicons?domain=arduino.cc 
  013. http://www.arduino.cc 
  014. http://www.arduino.cc/en/Guide/Introduction 
  015. https://github.com/arduino/Arduino/ 
  016. http://www.google.com/s2/favicons?domain=atlassian.com 
  017. https://www.atlassian.com/ 
  018. https://www.atlassian.com/company/about/values 
  019. https://s2.googleusercontent.com/s2/favicons?domain_url=https://axemsolutions.io/ 
  020. https://axemsolutions.io 
  021. https://axemsolutions.io/dem_doc/index.html 
  022. https://github.com/axem-solutions/dem 
  023. http://www.google.com/s2/favicons?domain=balsamiq.com 
  024. https://balsamiq.com/ 
  025. https://balsamiq.com/company/#goodcitizen 
  026. http://www.google.com/s2/favicons?domain=bevry.me 
  027. https://bevry.me 
  028. https://github.com/docpad/docpad 
  029. http://www.google.com/s2/favicons?domain=buffer.com 
  030. https://buffer.com/ 
  031. https://buffer.com/transparency 
  032. https://github.com/bufferapp/ 
  033. http://joel.is/why-we-have-a-core-value-of-transparency-at-our-startup/ 
  034. https://open.buffer.com/buffer-public-revenue-dashboard/ 
  035. http://www.google.com/s2/favicons?domain=codecombat.com/ 
  036. http://codecombat.com/ 
  037. http://codecombat.com/legal 
  038. https://github.com/codecombat/codecombat 
  039. http://blog.codecombat.com/why-you-should-open-source-your-startup 
  040. http://www.google.com/s2/favicons?domain=couchbase.com 
  041. http://www.couchbase.com/ 
  042. http://www.couchbase.com/open-source 
  043. https://github.com/couchbase 
  044. http://www.google.com/s2/favicons?domain=dangerousprototypes.com 
  045. http://dangerousprototypes.com/ 
  046. http://dangerousprototypes.com/docs/About 
  047. https://code.google.com/archive/p/dangerous-prototypes-open-hardware/ 
  048. http://www.google.com/s2/favicons?domain=documenso.com 
  049. https://documenso.com/ 
  050. https://documenso.com/blog/manifest 
  051. https://github.com/documenso/documenso 
  052. https://documenso.com/blog/pre-seed 
  053. https://documenso.com/open/ 
  054. http://www.google.com/s2/favicons?domain=dreamwidth.org 
  055. http://www.dreamwidth.org/ 
  056. http://www.dreamwidth.org/about 
  057. https://github.com/dreamwidth/ 
  058. http://www.google.com/s2/favicons?domain=elastic.co 
  059. https://elastic.co 
  060. https://www.elastic.co/about/free-and-open 
  061. https://github.com/elastic/ 
  062. http://www.google.com/s2/favicons?domain=gitlab.com 
  063. https://about.gitlab.com/ 
  064. https://about.gitlab.com/about/ 
  065. https://gitlab.com/groups/gitlab-org 
  066. http://www.google.com/s2/favicons?domain=gratipay.com 
  067. https://gratipay.com/ 
  068. http://inside.gratipay.com/big-picture/welcome 
  069. https://github.com/gratipay 
  070. https://github.com/gratipay/finances#readme 
  071. http://growstuff.org/assets/favicon-2f083c214b9adaf9e2ce78bcd532e4c9.ico 
  072. http://www.growstuff.org/ 
  073. http://web.archive.org/web/20150906064358/http://wiki.growstuff.org/index.php/Values 
  074. https://github.com/Growstuff/growstuff 
  075. http://blog.growstuff.org/2013/02/20/why-growstuff-is-open-source/ 
  076. http://www.google.com/s2/favicons?domain=lulzbot.com 
  077. https://lulzbot.com/ 
  078. https://en.wikipedia.org/wiki/Aleph_Objects 
  079. https://lulzbot.com/learn/choosing-a-free-license-for-your-3d-project 
  080. https://gitlab.com/lulzbot3d 
  081. http://www.google.com/s2/favicons?domain=mapbox.com 
  082. http://mapbox.com/ 
  083. http://mapbox.com/about/open 
  084. https://github.com/mapbox 
  085. http://www.google.com/s2/favicons?domain=neocities.org 
  086. https://neocities.org/ 
  087. http://wayback.archive.org/web/20150907143713id_/https://neocities.org/stats 
  088. https://github.com/neocities/neocities 
  089. https://neocities.org/blog/open-company-progress-report-2014 
  090. https://www.google.com/s2/favicons?domain=vector.im 
  091. https://vector.im/ 
  092. https://github.com/vector-im/ 
  093. https://www.google.com/s2/favicons?domain=niteo.co 
  094. https://niteo.co/ 
  095. https://github.com/niteoweb/handbook 
  096. https://github.com/paretoSecurity/ 
  097. http://www.google.com/s2/favicons?domain=opencraft.com 
  098. http://opencraft.com/ 
  099. https://www.google.com/s2/favicons?domain=protonmail.ch 
  100. https://protonmail.ch 
  101. https://protonmail.com/blog/protonmail-open-source/ 
  102. https://github.com/ProtonMail/WebClient 
  103. https://github.com/openpgpjs/openpgpjs 
  104. https://protonmail.com/blog/protonmail-open-source-crytography/ 
  105. http://www.google.com/s2/favicons?domain=readthedocs.org 
  106. https://readthedocs.org/ 
  107. https://docs.readthedocs.io/en/latest/open-source-philosophy.html 
  108. https://github.com/rtfd/readthedocs.org 
  109. http://blog.readthedocs.com/read-the-docs-2017-stats/#funding 
  110. http://www.google.com/s2/favicons?domain=redhat.com 
  111. https://www.redhat.com/ 
  112. https://www.redhat.com/en/jobs/life/culture 
  113. https://www.redhat.com/en/open-source/communities 
  114. http://www.google.com/s2/favicons?domain=rstudio.com 
  115. https://www.rstudio.com 
  116. https://www.rstudio.com/about/what-makes-rstudio-different/ 
  117. https://www.rstudio.com/about 
  118. https://www.google.com/s2/favicons?domain=sentry.io 
  119. https://sentry.io/ 
  120. https://sentry.io/about/ 
  121. https://github.com/getsentry/sentry 
  122. https://blog.sentry.io/2015/06/30/driven-by-open-source 
  123. https://www.google.com/s2/favicons?domain=softwaremill.com 
  124. https://softwaremill.com/ 
  125. https://blog.softwaremill.com/40-ceos-in-one-company-teal-organisation-interview-21124c34c16 
  126. https://github.com/softwaremill 
  127. https://softwaremill.com/finances-in-company/?utm_source=GitHub&utm_medium=teal 
  128. http://i.imgur.com/Xe9AYlw.png 
  129. https://tessel.io/ 
  130. https://tessel.io/opensource 
  131. https://github.com/tessel/project 
  132. https://www.google.com/s2/favicons?domain=transloadit.com 
  133. https://transloadit.com/ 
  134. https://transloadit.com/open-source/ 
  135. https://uppy.io 
  136. https://transloadit.com/jobs/ 
  137. https://www.google.com/s2/favicons?domain=wemake.services 
  138. https://wemake.services/ 
  139. https://github.com/wemake-services/meta/blob/master/README.md#purpose 
  140. https://github.com/wemake-services 
  141. https://www.google.com/s2/favicons?domain=watermelontools.com 
  142. https://www.watermelontools.com/ 
  143. https://www.watermelontools.com/post/promoting-openness-starting-with-the-codebase 
  144. https://github.com/watermelontools/watermelon 
  145. https://web.archive.org/web/20201130045805/https://www.clarify.so/ 
  146. https://clarifyhq.com/open 
  147. https://en.wikipedia.org/wiki/Wikinomics 
  148. https://www.redhat.com/en/explore/the-open-organization-book 
  149. https://opensource.com/open-organization/resources/field-guide 
  150. http://www.paulgraham.com/opensource.html 
  151. http://www.openp2pdesign.org/2011/open-design/business-models-for-open-hardware/ 
  152. http://www.rogerclarke.com/EC/Bled04.html 
  153. http://www.longtail.com/the_long_tail/2009/01/a-business-mode.html 
  154. http://www.economist.com/node/5624944 
  155. https://medium.com/gratipay-blog/the-second-open-company-4cbab7ca1a47 
  156. https://medium.com/open-companies/why-open-companies-fdb74d1b4f0f 
  157. http://www.opencompany.org/resources/whitepaper.pdf 
  158. https://www.youtube.com/watch?v=MAlnMWlvw9g 
  159. http://www.opencompany.org/directory/ 
  160. http://wayback.archive.org/web/20110824041839/http://www.openbusiness.cc/category/directory/openbusiness/ 
  161. http://wayback.archive.org/web/20120727175118/http://www.openbusiness.cc/open100/about/ 
  162. http://thevarguy.com/var-guy/var-guys-open-source-50 
  163. http://wayback.archive.org/web/20121118155240/http://www.thevarguy.com/the-open-source-50/the-open-source-50-listed-a-to-z/ 
  164. http://wayback.archive.org/web/20120509194329/http://www.thevarguy.com/the-open-source-50/the-open-source-50-a-to-z-2010-edition/ 
  165. http://thevarguy.com/open-source-application-software-companies/top-50-open-source-companies-where-are-they-now 
  166. https://github.com/paulhendricks/awesome-open 
  167. https://creativecommons.org/publicdomain/zero/1.0/ 
Checking URLs: ✓✓✓✓✓→→→xx✓✓→→✓→✓→✓✓✓→→✓→✓→→✓→→→→→→✓→→✓→✓→✓✓→→→→✓✓→✓→→→✓✓→✓→?✓✓✓✓→✓→✓✓✓✓→→→✓→x✓→✓→✓✓✓→→→→✓✓→✓✓→→→→✓→→→→→✓→→→→→✓→→✓→→→→→→→✓→→✓✓✓✓✓→✓✓→✓✓✓✓?✓✓✓✓✓✓→→→→✓→→→→✓→→→✓✓✓??✓→✓→→

Issues :-(
> Links 
  1. [L046] 403 http://www.arduino.cc  
  2. [L046] 403 http://www.arduino.cc/en/Guide/Introduction  
  3. [L059]  http://inside.gratipay.com/big-picture/welcome Failed to open TCP connection to inside.gratipay.com:80 (getaddrinfo: nodename nor servname provided, or not known) 
  4. [L060] 404 http://blog.growstuff.org/2013/02/20/why-growstuff-is-open-source/  
  5. [L082]  https://clarifyhq.com/open Failed to open TCP connection to clarifyhq.com:443 (getaddrinfo: nodename nor servname provided, or not known) 
  6. [L110]  http://thevarguy.com/var-guy/var-guys-open-source-50 Failed to open TCP connection to thevarguy.com:80 (getaddrinfo: nodename nor servname provided, or not known) 
  7. [L110]  http://thevarguy.com/open-source-application-software-companies/top-50-open-source-companies-where-are-they-now Failed to open TCP connection to thevarguy.com:80 (getaddrinfo: nodename nor servname provided, or not known) 
> Dupes 
  1. [L01] https://github.com/sindresorhus/awesome
  2. [L25] http://blog.gittip.com/post/26350459746/the-first-open-company/
  3. [L50] https://bevry.me
  4. [L64] https://vector.im/
  5. [L66] http://opencraft.com/

Wrote results to ab-results-README.md.json
Wrote filtered results to ab-results-README.md-filtered.json
Wrote markdown table results to ab-results-README.md-markdown-table.json
opencompany / awesome-open-company

Validate pull requests with Travis #23
