search

github / codeql-go

The CodeQL extractor and libraries for Go.
MIT License
465 stars 126 forks source link

Project: Taint-track all relevant packages from golang.org/x/* #471

Closed gagliardetto closed 3 years ago

gagliardetto commented 3 years ago

Project proposal: Taint-track the golang.org/x/* packages.

Here is a list of the **top 100 golang.org/x/*** packages sorted by how many times they were imported (taken from here)

A package is relevant if it has elements that can be put into a TaintTracking model. Not all packages from the below list will be "relevant" at the end.

I will use this ticket to track the progress.

Page 1

  1. [x] golang.org/x/net/context (docs)
    • Description: Package context defines the Context type, which carries deadlines, cancelation signals, and other request-scoped values across API boundaries and between processes.
    • Imported by: 39240
    • Comment: :red_circle: No apparent relevant elements.
  2. [ ] golang.org/x/oauth2 (docs)
    • Description: Package oauth2 provides support for making OAuth2 authorized and authenticated HTTP requests, as specified in RFC 6749.
    • Imported by: 10797
    • Comment: :red_circle: No apparent relevant elements.
  3. [ ] golang.org/x/sys/unix (docs)
    • Description: Package unix contains an interface to the low-level operating system primitives.
    • Imported by: 6584
    • Comment: :red_circle: No apparent relevant elements.
  4. [ ] golang.org/x/crypto/ssh (docs)
    • Description: Package ssh implements an SSH client and server.
    • Imported by: 4456
    • Comment: :orange_circle:
  5. [ ] golang.org/x/crypto/bcrypt (docs)
    • Description: Package bcrypt implements Provos and Mazières's bcrypt adaptive hashing algorithm.
    • Imported by: 4000
    • Comment: :orange_circle:
  6. [ ] golang.org/x/crypto/ssh/terminal (docs)
    • Description: Package terminal provides support functions for dealing with terminals, as commonly found on UNIX systems.
    • Imported by: 3615
    • Comment: :red_circle:
  7. [ ] golang.org/x/sync/errgroup (docs)
    • Description: Package errgroup provides synchronization, error propagation, and Context cancelation for groups of goroutines working on subtasks of a common task.
    • Imported by: 3514
    • Comment: :red_circle:
  8. [ ] golang.org/x/net/websocket (docs)
    • Description: Package websocket implements a client and server for the WebSocket protocol as specified in RFC 6455.
    • Imported by: 3192
    • Comment: :orange_circle:
  9. [ ] golang.org/x/crypto/sha3 (docs)
    • Description: Package sha3 implements the SHA-3 fixed-output-length hash functions and the SHAKE variable-output-length hash functions defined by FIPS-202.
    • Imported by: 3098
    • Comment: :red_circle:
  10. [ ] golang.org/x/oauth2/google (docs)
    • Description: Package google provides support for making OAuth2 authorized and authenticated HTTP requests to Google APIs.
    • Imported by: 3083
    • Comment: :red_circle:

Page 2

  1. [ ] golang.org/x/net/context/ctxhttp (docs)
    • Description: Package ctxhttp provides helper functions for performing context-aware HTTP requests.
    • Imported by: 2814
    • Comment: :red_circle:
  2. [ ] golang.org/x/net/html (docs)
    • Description: Package html implements an HTML5-compliant tokenizer and parser.
    • Imported by: 2369
    • Comment: :green_circle: :
  3. [ ] golang.org/x/crypto/pbkdf2 (docs)
    • Description: Package pbkdf2 implements the key derivation function PBKDF2 as defined in RFC 2898 / PKCS #5 v2.0.
    • Imported by: 2345
    • Comment: :red_circle:
  4. [ ] golang.org/x/text/transform (docs)
    • Description: Package transform provides reader and writer wrappers that transform the bytes passing through as well as various transformations.
    • Imported by: 2261
    • Comment: :green_circle:
  5. [ ] golang.org/x/xerrors (docs)
    • Description: Package xerrors implements functions to manipulate errors.
    • Imported by: 2224
    • Comment: :green_circle:
  6. [ ] golang.org/x/time/rate (docs)
    • Description: Package rate provides a rate limiter.
    • Imported by: 2165
    • Comment: :red_circle:
  7. [ ] golang.org/x/net/http2 (docs)
    • Description: Package http2 implements the HTTP/2 protocol.
    • Imported by: 1780
    • Comment: :green_circle:
  8. [ ] golang.org/x/crypto/ripemd160 (docs)
    • Description: Package ripemd160 implements the RIPEMD-160 hash algorithm.
    • Imported by: 1380
    • Comment: :red_circle:
  9. [ ] golang.org/x/text/encoding (docs)
    • Description: Package encoding defines an interface for character encodings, such as Shift JIS and Windows 1252, that can convert to and from UTF-8.
    • Imported by: 1333
    • Comment: :green_circle:
  10. [ ] golang.org/x/tools/imports (docs)
    • Description: Package imports implements a Go pretty-printer (like package "go/format") that also adds or removes import statements as necessary.
    • Imported by: 1157
    • Comment: :red_circle:

Page 3

  1. [ ] golang.org/x/text/language (docs)
    • Description: Package language implements BCP 47 language tags and related functionality.
    • Imported by: 1129
    • Comment: :red_circle:
  2. [ ] golang.org/x/crypto/scrypt (docs)
    • Description: Package scrypt implements the scrypt key derivation function as defined in Colin Percival's paper "Stronger Key Derivation via Sequential Memory-Hard Functions" (https://www.tarsnap.com/scrypt/scrypt.pdf).
    • Imported by: 999
    • Comment: :red_circle:
  3. [ ] golang.org/x/crypto/ed25519 (docs)
    • Description: Package ed25519 implements the Ed25519 signature algorithm.
    • Imported by: 994
    • Comment: :red_circle:
  4. [ ] golang.org/x/tools/go/analysis (docs)
    • Description: Package analysis defines the interface between a modular static analysis and an analysis driver program.
    • Imported by: 974
    • Comment: :red_circle:
  5. [ ] golang.org/x/text/unicode/norm (docs)
    • Description: Package norm contains types and functions for normalizing Unicode strings.
    • Imported by: 927
    • Comment: :green_circle:
  6. [ ] golang.org/x/tools/go/loader (docs)
    • Description: Package loader loads a complete Go program from source code, parsing and type-checking the initial packages plus their transitive closure of dependencies.
    • Imported by: 897
    • Comment: :red_circle:
  7. [ ] golang.org/x/tools/go/packages (docs)
    • Description: Package packages loads Go packages for inspection and analysis.
    • Imported by: 894
    • Comment: :red_circle:
  8. [ ] golang.org/x/image/font (docs)
    • Description: Package font defines an interface for font faces, for drawing text on an image.
    • Imported by: 795
    • Comment: :red_circle:
  9. [ ] golang.org/x/crypto/acme/autocert (docs)
    • Description: Package autocert provides automatic access to certificates from Let's Encrypt and any other ACME-based CA.
    • Imported by: 789
    • Comment: :red_circle:
  10. [ ] golang.org/x/text/encoding/charmap (docs)
    • Description: Package charmap provides simple character encodings such as IBM Code Page 437 and Windows 1252.
    • Imported by: 785
    • Comment: :green_circle:

Page 4

  1. [ ] golang.org/x/net/html/charset (docs)
    • Description: Package charset provides common text encodings for HTML documents.
    • Imported by: 781
    • Comment: :green_circle: :
  2. [ ] golang.org/x/net/proxy (docs)
    • Description: Package proxy provides support for a variety of protocols to proxy network data.
    • Imported by: 773
    • Comment: :red_circle: :
  3. [ ] golang.org/x/text/encoding/simplifiedchinese (docs)
    • Description: Package simplifiedchinese provides Simplified Chinese encodings such as GBK.
    • Imported by: 755
    • Comment: :red_circle:
  4. [ ] golang.org/x/net/publicsuffix (docs)
    • Description: Package publicsuffix provides a public suffix list based on data from https://publicsuffix.org/ A public suffix is one under which Internet users can directly register names.
    • Imported by: 745
    • Comment: :orange_circle: :
  5. [ ] golang.org/x/image/math/fixed (docs)
    • Description: Package fixed implements fixed-point integer types.
    • Imported by: 743
    • Comment: :red_circle: :
  6. [ ] golang.org/x/tools/go/ast/astutil (docs)
    • Description: Package astutil contains common utilities for working with the Go AST.
    • Imported by: 740
    • Comment: :red_circle: :
  7. [ ] golang.org/x/crypto/nacl/secretbox (docs)
    • Description: Package secretbox encrypts and authenticates small messages.
    • Imported by: 724
    • Comment: :green_circle: :
  8. [ ] golang.org/x/sys/windows (docs)
    • Description: Package windows contains an interface to the low-level operating system primitives.
    • Imported by: 722
    • Comment: :orange_circle: :
  9. [ ] golang.org/x/crypto/blake2b (docs)
    • Description: Package blake2b implements the BLAKE2b hash algorithm defined by RFC 7693 and the extendable output function (XOF) BLAKE2Xb.
    • Imported by: 720
    • Comment: :red_circle: :
  10. [ ] golang.org/x/net/html/atom (docs)
    • Description: Package atom provides integer codes (also known as atoms) for a fixed set of frequently occurring HTML strings: tag names and attribute keys such as "p" and "id".
    • Imported by: 697
    • Comment: :red_circle: :

Page 5

  1. [ ] golang.org/x/crypto/curve25519 (docs)
    • Description: Package curve25519 provides an implementation of the X25519 function, which performs scalar multiplication on the elliptic curve known as Curve25519.
    • Imported by: 687
    • Comment: :red_circle: :
  2. [ ] golang.org/x/net/ipv4 (docs)
    • Description: Package ipv4 implements IP-level socket options for the Internet Protocol version 4.
    • Imported by: 679
    • Comment: :orange_circle: :
  3. [ ] golang.org/x/sys/cpu (docs)
    • Description: Package cpu implements processor feature detection for various CPU architectures.
    • Imported by: 673
    • Comment: :red_circle: :
  4. [ ] golang.org/x/oauth2/jwt (docs)
    • Description: Package jwt implements the OAuth 2.0 JSON Web Token flow, commonly known as "two-legged OAuth 2.0".
    • Imported by: 638
    • Comment: :red_circle: :
  5. [ ] golang.org/x/text/encoding/unicode (docs)
    • Description: Package unicode provides Unicode encodings such as UTF-16.
    • Imported by: 634
    • Comment: :red_circle: :
  6. [ ] golang.org/x/net/idna (docs)
    • Description: Package idna implements IDNA2008 using the compatibility processing defined by UTS (Unicode Technical Standard) #46, which defines a standard to deal with the transition from IDNA2003.
    • Imported by: 628
    • Comment: :green_circle: :
  7. [ ] golang.org/x/crypto/openpgp (docs)
    • Description: Package openpgp implements high level operations on OpenPGP messages.
    • Imported by: 607
    • Comment: :green_circle: :
  8. [ ] golang.org/x/net/trace (docs)
    • Description: Package trace implements tracing of requests and long-lived objects.
    • Imported by: 599
    • Comment: :orange_circle: :
  9. [ ] golang.org/x/tools/go/ssa (docs)
    • Description: Package ssa defines a representation of the elements of Go programs (packages, types, functions, variables and constants) using a static single-assignment (SSA) form intermediate representation (IR) for the bodies of functions.
    • Imported by: 591
    • Comment: :red_circle: :
  10. [ ] golang.org/x/sync/semaphore (docs)
    • Description: Package semaphore provides a weighted semaphore implementation.
    • Imported by: 588
    • Comment: :red_circle: :

Page 6

  1. [ ] golang.org/x/crypto/ssh/agent (docs)
    • Description: agent implements the ssh-agent protocol, and provides both a client and a server.
    • Imported by: 574
    • Comment: :red_circle: :
  2. [ ] golang.org/x/crypto/hkdf (docs)
    • Description: Package hkdf implements the HMAC-based Extract-and-Expand Key Derivation Function (HKDF) as defined in RFC 5869.
    • Imported by: 566
    • Comment: :red_circle: :
  3. [ ] golang.org/x/tools/go/types/typeutil (docs)
    • Description: typeutil defines various utilities for types, such as Map, a mapping from types.Type to interface{} values.
    • Imported by: 563
    • Comment: :red_circle:
  4. [ ] golang.org/x/crypto/chacha20poly1305 (docs)
    • Description: Package chacha20poly1305 implements the ChaCha20-Poly1305 AEAD and its extended nonce variant XChaCha20-Poly1305, as specified in RFC 8439 and draft-irtf-cfrg-xchacha-01.
    • Imported by: 525
    • Comment: :red_circle:
  5. [ ] golang.org/x/net/http2/hpack (docs)
    • Description: hpack implements HPACK, a compression format for efficiently representing HTTP header fields in the context of HTTP/2.
    • Imported by: 495
    • Comment: :green_circle:
  6. [ ] golang.org/x/tools/go/analysis/passes/inspect (docs)
    • Description: an Analyzer that provides an AST inspector (golang.org/x/tools/go/ast/inspect.Inspect) for the syntax trees of a package.
    • Imported by: 495
    • Comment: :red_circle: :
  7. [ ] golang.org/x/tools/go/ast/inspector (docs)
    • Description: inspector provides helper functions for traversal over the syntax trees of a package, including node filtering by type, and materialization of the traversal stack.
    • Imported by: 494
    • Comment: :red_circle:
  8. [ ] golang.org/x/crypto/openpgp/errors (docs)
    • Description: errors contains common error types for the OpenPGP packages.
    • Imported by: 483
    • Comment: :red_circle:
  9. [ ] golang.org/x/sync/syncmap (docs)
    • Description: Package syncmap provides a concurrent map implementation.
    • Imported by: 481
    • Comment: :red_circle:
  10. [ ] golang.org/x/oauth2/clientcredentials (docs)
    • Description: Package clientcredentials implements the OAuth2.0 "client credentials" token flow, also known as the "two-legged OAuth 2.0".
    • Imported by: 474
    • Comment: :red_circle:

Page 7

  1. [ ] golang.org/x/text/encoding/japanese (docs)
    • Description: Package japanese provides Japanese encodings such as EUC-JP and Shift JIS.
    • Imported by: 457
    • Comment: :red_circle: :
  2. [ ] golang.org/x/crypto/openpgp/armor (docs)
    • Description: Package armor implements OpenPGP ASCII Armor, see RFC 4880.
    • Imported by: 446
    • Comment: :red_circle:
  3. [ ] golang.org/x/net/ipv6 (docs)
    • Description: Package ipv6 implements IP-level socket options for the Internet Protocol version 6.
    • Imported by: 420
    • Comment: :orange_circle:
  4. [ ] golang.org/x/net/netutil (docs)
    • Description: Package netutil provides network utility functions, complementing the more common ones in the net package.
    • Imported by: 416
    • Comment: :orange_circle:
  5. [ ] golang.org/x/crypto/openpgp/packet (docs)
    • Description: Package packet implements parsing and serialization of OpenPGP packets, as specified in RFC 4880.
    • Imported by: 408
    • Comment: :orange_circle:
  6. [ ] golang.org/x/text/encoding/traditionalchinese (docs)
    • Description: Package traditionalchinese provides Traditional Chinese encodings such as Big5.
    • Imported by: 389
    • Comment: :red_circle:
  7. [ ] golang.org/x/crypto/nacl/box (docs)
    • Description: Package box authenticates and encrypts small messages using public-key cryptography.
    • Imported by: 381
    • Comment: :red_circle:
  8. [ ] golang.org/x/text/encoding/korean (docs)
    • Description: Package korean provides Korean encodings such as EUC-KR.
    • Imported by: 381
    • Comment: :red_circle:
  9. [ ] golang.org/x/crypto/blowfish (docs)
    • Description: Package blowfish implements Bruce Schneier's Blowfish encryption algorithm.
    • Imported by: 358
    • Comment: :red_circle:
  10. [ ] golang.org/x/crypto/pkcs12 (docs)
    • Description: Package pkcs12 implements some of PKCS#12.
    • Imported by: 356
    • Comment: :red_circle:

Page 8

  1. [ ] golang.org/x/crypto/salsa20/salsa (docs)
    • Description: Package salsa provides low-level access to functions in the Salsa family.
    • Imported by: 356
    • Comment: :red_circle:
  2. [ ] golang.org/x/tools/go/buildutil (docs)
    • Description: Package buildutil provides utilities related to the go/build package in the standard library.
    • Imported by: 355
    • Comment: :red_circle:
  3. [ ] golang.org/x/tools/godoc/vfs (docs)
    • Description: defines types for abstract file system access and provides an implementation accessing the file system of the underlying OS.Description:
    • Imported by: 350
    • Comment: :orange_circle:
  4. [ ] golang.org/x/image/bmp (docs)
    • Description: Package bmp implements a BMP image decoder and encoder.
    • Imported by: 279
    • Comment: :red_circle:
  5. [ ] golang.org/x/sync/singleflight (docs)
    • Description: Package singleflight provides a duplicate function call suppression mechanism.
    • Imported by: 279
    • Comment: :red_circle:
  6. [ ] golang.org/x/tools/go/ssa/ssautil (docs)
    • Description:
    • Imported by: 278
    • Comment: :red_circle:
  7. [ ] golang.org/x/crypto/argon2 (docs)
    • Description: Package argon2 implements the key derivation function Argon2.
    • Imported by: 276
    • Comment: :red_circle:
  8. [ ] golang.org/x/tools/go/callgraph (docs)
    • Description: Package callgraph defines the call graph and various algorithms and utilities to operate on it.
    • Imported by: 274
    • Comment: :red_circle:
  9. [ ] golang.org/x/net/http/httpguts (docs)
    • Description: Package httpguts provides functions implementing various details of the HTTP specification.
    • Imported by: 273
    • Comment: :green_circle: :
  10. [ ] golang.org/x/crypto/ocsp (docs)
    • Description: Package ocsp parses OCSP responses as specified in RFC 2560.
    • Imported by: 272
    • Comment: :green_circle: :

Page 9

  1. [ ] golang.org/x/crypto/poly1305 (docs)
  2. [ ] golang.org/x/oauth2/jws (docs)
    • Description: Package jws provides a partial implementation of JSON Web Signature encoding and decoding.
    • Imported by: 257
    • Comment: :orange_circle:
  3. [ ] golang.org/x/net/icmp (docs)
    • Description: Package icmp provides basic functions for the manipulation of messages used in the Internet Control Message Protocols, ICMPv4 and ICMPv6.
    • Imported by: 254
    • Comment: :orange_circle: :
  4. [ ] golang.org/x/image/colornames (docs)
    • Description: Package colornames provides named colors as defined in the SVG 1.1 spec.
    • Imported by: 251
    • Comment: :red_circle:
  5. [ ] golang.org/x/text/message (docs)
    • Description: Package message implements formatted I/O for localized strings with functions analogous to the fmt's print functions.
    • Imported by: 249
    • Comment: :green_circle: :
  6. [ ] golang.org/x/crypto/md4 (docs)
    • Description: Package md4 implements the MD4 hash algorithm as defined in RFC 1320.
    • Imported by: 247
    • Comment: :red_circle:
  7. [ ] golang.org/x/image/draw (docs)
    • Description: Package draw provides image composition functions.
    • Imported by: 239
    • Comment: :red_circle:
  8. [ ] golang.org/x/sys/windows/svc (docs)
    • Description: Package svc provides everything required to build Windows service.
    • Imported by: 223
    • Comment: :red_circle:
  9. [ ] golang.org/x/text/runes (docs)
    • Description: Package runes provide transforms for UTF-8 encoded text.
    • Imported by: 216
    • Comment: :orange_circle: :
  10. [ ] golang.org/x/net/bpf (docs)
    • Description: Package bpf implements marshaling and unmarshaling of programs for the Berkeley Packet Filter virtual machine, and provides a Go implementation of the virtual machine.
    • Imported by: 206
    • Comment: :red_circle:

Page 10

  1. [ ] golang.org/x/crypto/cast5 (docs)
    • Description: Package cast5 implements CAST5, as defined in RFC 2144.
    • Imported by: 206
    • Comment: :red_circle:
  2. [ ] golang.org/x/mobile/event/size (docs)
    • Description: Package size defines an event for the dimensions, physical resolution and orientation of the app's window.
    • Imported by: 201
    • Comment: :red_circle:
  3. [ ] golang.org/x/text/encoding/htmlindex (docs)
    • Description: Package htmlindex maps character set encoding names to Encodings as recommended by the W3C for use in HTML 5.
    • Imported by: 198
    • Comment: :red_circle:
  4. [ ] golang.org/x/image/tiff (docs)
    • Description: Package tiff implements a TIFF image decoder and encoder.
    • Imported by: 193
    • Comment: :red_circle:
  5. [ ] golang.org/x/tools/go/analysis/singlechecker (docs)
    • Description: Package singlechecker defines the main function for an analysis driver with only a single analysis.
    • Imported by: 179
    • Comment: :red_circle:
  6. [ ] golang.org/x/crypto/openpgp/s2k (docs)
    • Description: Package s2k implements the various OpenPGP string-to-key transforms as specified in RFC 4800 section 3.7.1.
    • Imported by: 179
    • Comment: :red_circle:
  7. [ ] golang.org/x/net/webdav (docs)
    • Description: Package webdav provides a WebDAV server implementation.
    • Imported by: 178
    • Comment: :red_circle:
  8. [ ] golang.org/x/crypto/acme (docs)
    • Description: Package acme provides an implementation of the Automatic Certificate Management Environment (ACME) spec.
    • Imported by: 176
    • Comment: :red_circle:
  9. [ ] golang.org/x/oauth2/github (docs)
    • Description: Package github provides constants for using OAuth2 to access Github.
    • Imported by: 174
    • Comment: :red_circle:
  10. [ ] golang.org/x/image/math/f64 (docs)
    • Description: Package f64 implements float64 vector and matrix types.
    • Imported by: 170
    • Comment: :red_circle:
smowton commented 3 years ago

@gagliardetto before investing effort in this, I would ask the question of each package: considering the sorts of data flow that CodeQL is used to search for (usually, remote / externally-controlled data -> exploitable parser or routine), how plausible is it that one of these packages might ever feature in that path? In particular that would mean libraries used to implement other libraries (e.g., http2?) are very unlikely to be good candidates, because the source and sink are likely to be in an application codebase, meaning only libraries directly used by an application would be good candidates.

Parts of golang.org/x/text seem the most plausible to me, but I haven't gone over them in detail -- I don't have a concrete plan right now, just suggesting giving them that quick plausibility test before investing much effort.

gagliardetto commented 3 years ago

@smowton I thought the same thing: I'll do an initial evaluation to remove unsuitable packages, and then later tackle the list of what remains.

I then changed my mind because 90% of the work that will need to be put into this project is the actual evaluation (I won't write a single line of code; not codeql, nor go).

Doing an initial evaluation, and then a second evaluation for when I will create the models is double work.

The most time and effort efficient way seems to me is going into the raw (above) list and doing the evaluation on the spot, and then immediately creating the models.

gagliardetto commented 3 years ago

@smowton You're right. A global initial evaluation is actually better.

I put a :red_circle: for packages I don't believe contain any valuable elements for TaintTracking or UntrustedFlowSources; packages that might contain something, but are either very big or ambiguous got a :orange_circle: ; packages I believe should be modeled got a :green_circle: .

I'll take another look in a few days and see whether I spot something else.

gagliardetto commented 3 years ago

To be honest, I thought I would find more modelable stuff in /x :unamused:

https://github.com/github/codeql-go/pull/438 keeps its place as my next big project :blush:

gagliardetto commented 3 years ago

Closing for future reopening.