proposal: cross-platform network frame communications package

[swbo97]

Proposal Details

Proposal: Cross-Platform Network Frame Communications Package

Summary

I propose the development of a new Go standard package to support cross-platform network frame communications, covering protocols such as Ethernet, ARP (Address Resolution Protocol), NDP (Neighbor Discovery Protocol), IPv4, and IPv6. This package aims to provide a robust, standardized solution for low-level network operations within the Go ecosystem.

Rationale

The Go standard library currently lacks comprehensive support for low-level network frame communications. While third-party libraries exist, they often fall short in terms of cross-platform compatibility, robustness, and maintenance. A standardized package would fill this gap, providing a consistent and reliable toolset for network programming directly within the Go language.

Importance

1. Simplified Development: A dedicated package will streamline the development process for applications requiring low-level network access, reducing reliance on external dependencies.
2. Cross-Platform Consistency: Ensuring compatibility across major operating systems (Linux, Windows, macOS) will enhance the portability of Go applications.
3. Enhanced Security: A well-maintained, officially supported package will enable timely updates and adherence to best security practices.
4. Community Growth: Addressing this common need can attract more developers to the Go community, fostering innovation and growth.

Proposal

• Package Name: netframes (or an equivalent name indicating network frame handling)
• Core Features:
  • Ethernet: APIs to construct, parse, and manipulate Ethernet frames for link-layer communication.
  • ARP: Functions for handling ARP requests and replies, essential for mapping network layer addresses to link-layer addresses.
  • NDP: Support for Neighbor Discovery Protocol functionalities in IPv6, crucial for address autoconfiguration and neighbor discovery.
  • IPv4/IPv6: Comprehensive tools for constructing, parsing, and manipulating IPv4 and IPv6 packets, including handling headers, fragmentation, and payloads.

Scope

1. Ethernet Frame Handling: Creation, parsing, and manipulation of Ethernet frames to facilitate link-layer communication.
2. ARP Support: Mechanisms to send and receive ARP requests and replies for address resolution.
3. NDP Integration: Implementation of NDP functionalities to support IPv6 networking operations.
4. IPv4/IPv6 Packet Management: Full support for building, parsing, and managing IPv4 and IPv6 packets, including headers and payloads.

Potential Contributions

I am willing to contribute significantly to this proposal if it is adopted. This includes developing and maintaining the package, ensuring cross-platform compatibility, and providing comprehensive documentation and examples.

Advantages and Applications

1. Network Security: Building tools for network security analysis and intrusion detection systems.
2. Software-Defined Networking (SDN): Developing SDN tools for dynamic network management and automation.
3. Network Diagnostics: Creating diagnostic tools for network monitoring and troubleshooting.

Technical Requirements

• libpcap Dependency: This standard package should rely on libpcap to ensure robust and efficient packet capturing and injection capabilities across different platforms. Libpcap is widely used and provides the necessary low-level access to network frames, making it an ideal foundation for this package.

By addressing these needs, this package can significantly enhance Go's network programming capabilities, making it a powerful tool for developers working on advanced networking applications.

[ianlancetaylor]

There seems to be some overlap with the x/net package here.

It's not clear to me why this should be in the standard library. https://go.dev/doc/faq#x_in_std . These seem fairly special purpose. Certainly we should have packages for these, but why in the standard library?

Note in particular that the standard library can't depend on libpcap. More generally the standard library can not depend on any code that is not written in Go.

[swbo97]

Thanks for pointing out the overlap with x/net, ianlancetaylor. My proposal focuses on lower-level network frame construction, parsing, and manipulation, specifically at the link layer and network layer, complementing x/net's higher-level functionalities.

This package can be a cornerstone for building secure network applications. It would enable functionalities like deep packet inspection for firewalls and IDS, as well as customizable network filters for specific security needs.

Use cases examples: Intrusion Detection System (IDS) - SDN - Network monitoring - Custom Network Protocols Development - Network Firmware equipments development

I understand the libpcap concern. We can explore pure Go libraries for packet capturing. As a Go network expert, I'm also open to exploring alternative approaches like a minimal Go implementation for essential packet handling needs, ensuring the package remains self-contained within the standard library.

I am committed to actively engaging with the community to refine this proposal if adopted. I welcome feedback from other developers who have experience with low-level network programming and those who might benefit from such a package. This collaborative effort will help us address any potential issues early on and ensure that the package meets the high standards of the Go project.

Developers working on high-level network protocols, such as HTTP (mainly for APIs development), benefit greatly from having reliable, well-maintained libraries in the standard library. This prevents them from needing to reinvent the wheel each time they implement HTTP functionalities. Similarly, low-level network programmers require standardized tools to avoid duplicating efforts and to ensure robust implementations.

Including such utilities in the standard library ensures a unified development experience for Go programmers. By providing a consistent API for low-level network operations, developers can rely on familiar and well-documented tools without needing to integrate disparate third-party libraries. This cohesion simplifies the learning curve for new developers and enhances productivity for experienced programmers, allowing them to focus on building innovative network solutions rather than managing dependencies and compatibility issues.

Thanks again for your feedback! I'm open to further discussion on how to best approach this, considering the Go standard library's needs and the developer community's benefit.

[swbo97]

Design Example:

// Frame represents a network frame with various protocol headers
type Frame struct {
    Eth   *Ethernet
    Arph  *ARP
    Iph   *IP
    Icmph *ICMP
    // Add other headers as needed
}

// NewFrame initializes and returns a new Frame
func NewFrame() *Frame {
    return &Frame{
        Eth:   NewEthernetHeader(),
        Arph:  NewARPHeader(),
        Iph:   NewIPv4Header(),
        Icmph: NewICMPHeader(),
    }
}

// FrameBytes concatenates and returns the byte representation of the Frame
func (frame *Frame) FrameBytes() []byte {
    sRValue := reflect.ValueOf(frame).Elem()
    sRType := sRValue.Type()
    var frameBytes []byte

    for i := 0; i < sRType.NumField(); i++ {
        value := sRValue.Field(i).Interface()
        switch sRType.Field(i).Name {
        case "Eth":
            if eth, ok := value.(*Ethernet); ok {
                frameBytes = append(frameBytes, eth.ToBytes()...)
            }
        case "Arph":
            if arph, ok := value.(*ARP); ok {
                frameBytes = append(frameBytes, arph.ToBytes()...)
            }
        case "Iph":
            if iph, ok := value.(*IP); ok {
                frameBytes = append(frameBytes, iph.ToBytes()...)
            }
        case "Icmph":
            if icmph, ok := value.(*ICMP); ok {
                frameBytes = append(frameBytes, icmph.ToBytes()...)
            }
        }
    }

    // Concatenate all byte slices into one
    var tmp []byte
    for _, s := range frameBytes {
        tmp = append(tmp, s)
    }
    return tmp
}
================================================================
type Ethernet struct {
    DestMacAddress   []byte
    SourceMacAddress []byte
    Dot1Q            []byte
    Type             []byte
}
func NewEthHeader() (Eth_header *Ethernet) {
    return &Ethernet{}
}
func (Header *Ethernet) BuildHeader(destmac string, src string, ntype uint16) { // }
func (Header *Ethernet) EthernetBytes() []byte { // }
func (Header *Ethernet) ParseEthernet(byte_slice []byte, isTrunked bool) { // }
func (Header *Ethernet) TagDot1Q(VlanID int64, Priority int64) { // }
func (Header *Ethernet) GetVlanID() (vlanid int64) { // }
================================================================
type ARP struct {
    HardwareType     []byte
    ProtocolType     []byte
    HardwareSize     []byte
    ProtocolSize     []byte
    Operation        []byte
    SenderMACaddress []byte
    SenderIPaddress  []byte
    TargetMACaddress []byte
    TargetIPaddress  []byte
}

func NewArpHeader() *ARP {
    return &ARP{}
}
func (Arp *ARP) BuildARPHeader(HardwareType uint16, ProtocolType uint16, SenderIP string, TargetIP string, SenderMACaddress string, TargetMACaddress string, Operation uint16) { // }
func (arp *ARP) ARPBytes() []byte { // }
func (arp *ARP) GetTargetMAC() []byte { // }
func (arp *ARP) SetTargetMAC(MAC []byte) { // }
func (arp *ARP) ParseARP(arp_byte_slice []byte) { // }
================================================================
type IP struct {
    Version_HL      []byte
    DS              []byte
    Total_Length    []byte
    Identification  []byte
    Flags           []byte
    TTL             []byte
    Protocol        []byte
    Header_checksum []byte
    Src_address     []byte
    Dst_address     []byte
}

type Packet struct {
    IPH  *IP
    Data []byte
}

func NewIpv4Header() (header *IP) {
    return &IP{}
}
func (Ip *IP) BuildIPV4Header(IPsrc string, IPdest string, Protocol uint8) { // }
func (Header *IP) IPV4Bytes() []byte { // }
func Fragment(data []byte, MTU int, Protocol uint8, IPsrc string, IPdest string) (Packets map[int]Packet) { // }
func (Header *IP) HeaderChecksum() { // }
func (Header *IP) SetSRC(ipstring string) { // }
func (Header *IP) SetDST(ipstring string) { // }
func (Header *IP) ReverseSrc() { // }
func (Header *IP) GetProtocol() uint8 { // }
func (Header *IP) SetDS(label uint8) { // }
func (Header *IP) ParseIPV4(byte_slice []byte) { // }
================================================================
etc  ...

// Example Ethernet Frame 
package main

import (
    "fmt"
    "netframes/protocols"
    "netframes/protocols/constfields"
)

func main() {
    frame := protocols.NewFrame()
    // BuildHeader func args: DSTMAC, SRCMAC, Protocol type
    frame.Eth.BuildHeader("ff:ff:ff:ff:ff:ff", "08:00:27:dd:c1:1f", constfields.TypeIPv4)
    // Optional: For IEEE 802.1q Tagging
    frame.Eth.TagDot1Q(102, 6) // args: VlanID and 802.1p value
    frameBytes := frame.FrameBytes()
    fmt.Printf("%x", frameBytes)
}

================================================================
// Example ARP Request/Reply
package main

import (
    "fmt"
    "netframes/protocols"
    "netframes/protocols/constfields"
)

func main() {
    frame := protocols.NewFrame()
    frame.Eth.BuildHeader("ff:ff:ff:ff:ff:ff", "08:00:27:dd:c1:1f", constfields.TypeARP)
    // ARP(Proto: IPv4 only supported) Func Args: srcIP, desIP, srcMAC, dstMAC, operation
    // ARP REQUEST (for reply: last func arg: constfields.ARP_Operation_Reply)
    frame.Arph.BuildARPHeader(constfields.HardwareTypeEthernet, constfields.TypeIPv4, "192.168.1.14", 
        "192.168.1.222", "08:00:27:dd:c1:1f", "00:00:00:00:00:00", constfields.ARP_OperationRequest)
    frameBytes := frame.FrameBytes()
    fmt.Printf("%x", frameBytes)
}

================================================================
// Example ICMPv4 Echo/Reply
package main

import (
    "fmt"
    "netframes/protocols"
    "netframes/protocols/constfields"
)

func main() {
    frame := protocols.NewFrame()
    frame.Eth.BuildHeader("08:00:27:ff:23:22", "08:00:27:dd:c1:1f", constfields.TypeIPv4)
    frame.Iph.BuildIPv4Header("192.168.1.14", "192.168.1.11", constfields.TypeICMP)
    // BuildICMPHeader func args: ICMP Message type
    frame.Icmph.BuildICMPHeader(constfields.ICMPTypeReply)
    frameBytes := frame.FrameBytes()
    fmt.Printf("%x", frameBytes)
}

================================================================
// Example IPv4 Fragmentation
package main

import (
    "fmt"
    "netframes/protocols"
    "netframes/protocols/constfields"
)

func main() {
    data := make([]byte, 8900) // 8900 bytes of Data
    frame := protocols.NewFrame()
    frame.Eth.BuildHeader("ff:ff:ff:ff:ff:ff", "08:00:27:dd:c1:1f", constfields.TypeIPv4)
    // Fragment func args: Data byte slice, MTU, IP Higher Protocol, IPSRC, IPDST
    packets := protocols.Fragment(data, 1500, constfields.TypeTCP, "192.168.0.12", "8.8.8.8")
    for _, v := range packets {
        frame.Iph = v
        frameBytes := frame.FrameBytes()
        fmt.Printf("%x", frameBytes)
    }
}
================================================================
// Example ending and handling frames
package main

import (
    "netframes/protocols"
    "netframes/protocols/constfields"
    "netframes/transport"
)

func handleFrames(senderReceiver transport.FrameHandler) {
    chn := make(chan protocols.Frame)
    go func() {
        err := senderReceiver.ReceiveFrame(1024, chn)
        if err != nil {
            panic(err)
        }
    }()

    for s := range chn {
        // Example Replying to an ICMP Echo request
        if s.Icmph != nil && s.Icmph.GetType() == constfields.ICMPTypeEcho {
            s.Icmph.BuildICMPHeader(constfields.ICMPTypeReply)
            s.Iph.ReverseSrc()
            _, err := senderReceiver.SendFrame(s.FrameBytes())
            if err != nil {
                panic(err)
            }
        }
    }
}

func main() {
    interfaceName := "eth0"
    os := "linux"

    senderReceiver, err := transport.NewFrameHandler(interfaceName, os)
    if err != nil {
        panic(err)
    }
    handleFrames(senderReceiver)
}

[gophun]

@sabouaram Please don't let your designs and responses be generated by AI. Nobody wants to argue with a machine.

[swbo97]

@gophun lol :-), I maybe used AI to reformulate in English what i want to say here but if you think that the idea and the code design is made by AI you are wrong ! The code design is inspired by a personal pkg GoNetDev (which is private now ). Second point, if you have some important things to say about the proposal you are the most welcome elsewhere keep your comments to yourself. Third point, just disliking without giving a fuck is really weird, say your opinion clearly and let us discuss ....

[bjorndm]

We should not rely on a C library and in stead port libpcap to Go completely.