search

anitsh / til

Today I Learn (til) - Github `Issues` used as daily learning management system for taking notes and storing resource links.
https://anitshrestha.com.np
MIT License
76 stars 11 forks source link

Systems Communication #448

Open anitsh opened 3 years ago

anitsh commented 3 years ago

Systems Communication

When we think of it as natural thing we do in our daily life, we can better paint how the systems communicate as we build system that communicates to fuction.

image

Data Exchange

Data exchange is the process of taking data structured under a source schema and transforming it into a target schema, so that the target data is an accurate representation of the source data. Data exchange allows data to be shared between different computer programs.

Partial list of popular generic languages used for data exchange in multiple domains.  Protocol Schemas Flexible Semantic verification Dictionary Information Model Synonyms and homonyms Dialecting Web standard Transformations Lightweight Human readable Compatibility
RDF Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Partial Subset of Semantic web
REBOL Yes Yes No Yes No Yes Yes No Yes Yes Yes  
Gellish Yes Yes Yes Yes No Yes Yes ISO No Yes Partial SQL, RDF/XML, OWL
Atom Yes Un­known Un­known Un­known No Un­known Yes Yes Yes No No XML dialect
JSON No Un­known Un­known Un­known No Un­known No Yes No Yes Yes subset of YAML
YAML No Un­known Un­known Un­known No Un­known No No No Yes Yes superset of JSON
XML Yes Yes No No No No Yes Yes Yes No Yes subset of SGML, HTML

Nomenclature

  • Schemas - Whether the language definition is available in a computer interpretable form
  • Flexible - Whether the language enables extension of the semantic expression capabilities without modifying the schema
  • Semantic verification - Whether the language definition enables semantic verification of the correctness of expressions in the language
  • Dictionary-Taxonomy - Whether the language includes a dictionary and a taxonomy (subtype-supertype hierarchy) of concepts with inheritance
  • Synonyms and homonyms - Whether the language includes and supports the use of synonyms and homonyms in the expressions
  • Dialecting - Whether the language definition is available in multiple natural languages or dialects
  • Web or ISO standard - Organization that endorsed the language as a standard
  • Transformations - Whether the language includes a translation to other standards
  • Lightweight - Whether a lightweight version is available, in addition to a full version
  • Human-readable - Whether expressions in the language are human-readable (readable by humans without training)
  • Compatibility - Which other tools are possible to use or required when using the language

Resource

anitsh commented 3 years ago

POSIX

image Standards are what make my computer able to communicate with yours, they make websites render the same on our respective computers, they allow our computers to interpret a live video feed do that we all see the same thing. But standard extend beyond the sharing of content between two computers, some standards define specific patterns that allow much more than network or file interoperability. POSIX is one such standard.

Linux is known to support most of the POSIX system call interfaces as well as a large extension to the POSIX specification, known as the Linux Standard Base, which aims to unify Linux distributions in both code and binary support. https://pubs.opengroup.org/onlinepubs/9699919799/idx/functions.html

Resource

anitsh commented 3 years ago

Inter-Process Communication (IPC)

In computer science, inter-process communication or interprocess communication (IPC) refers specifically to the mechanisms an operating system provides to allow the processes to manage shared data. Typically, applications can use IPC, categorized as clients and servers, where the client requests data and the server responds to client requests. Many applications are both clients and servers, as commonly seen in distributed computing.

IPC is very important to the design process for microkernels and nanokernels, which reduce the number of functionalities provided by the kernel. Those functionalities are then obtained by communicating with servers via IPC, leading to a large increase in communication when compared to a regular monolithic kernel. IPC interfaces generally encompass variable analytic framework structures. These processes ensure compatibility between the multi-vector protocols upon which IPC models rely.

An IPC mechanism is either synchronous or asynchronous. Synchronization primitives may be used to have synchronous behavior with an asynchronous IPC mechanism.

Approaches

Different approaches to IPC have been tailored to different software requirements, such as performance, modularity, and system circumstances such as network bandwidth and latency.

Method Short Description Provided by (operating systems or other environments)
File A record stored on disk, or a record synthesized on demand by a file server, which can be accessed by multiple processes. Most operating systems
Communications file A unique form of IPC in the late-1960s that most closely resembles Plan 9's 9P protocol Dartmouth Time-Sharing System
Signal; also Asynchronous System Trap A system message sent from one process to another, not usually used to transfer data but instead used to remotely command the partnered process. Most operating systems
Socket Data sent over a network interface, either to a different process on the same computer or to another computer on the network. Stream-oriented (TCP; data written through a socket requires formatting to preserve message boundaries) or more rarely message-oriented (UDP, SCTP). Most operating systems
Unix domain socket Similar to an internet socket, but all communication occurs within the kernel. Domain sockets use the file system as their address space. Processes reference a domain socket as an inode, and multiple processes can communicate with one socket All POSIX operating systems and Windows 10[3]
Message queue A data stream similar to a socket, but which usually preserves message boundaries. Typically implemented by the operating system, they allow multiple processes to read and write to the message queue without being directly connected to each other. Most operating systems
Anonymous pipe A unidirectional data channel using standard input and output. Data written to the write-end of the pipe is buffered by the operating system until it is read from the read-end of the pipe. Two-way communication between processes can be achieved by using two pipes in opposite "directions". All POSIX systems, Windows
Named pipe A pipe that is treated like a file. Instead of using standard input and output as with an anonymous pipe, processes write to and read from a named pipe, as if it were a regular file. All POSIX systems, Windows, AmigaOS 2.0+
Shared memory Multiple processes are given access to the same block of memory, which creates a shared buffer for the processes to communicate with each other. All POSIX systems, Windows
Message passing Allows multiple programs to communicate using message queues and/or non-OS managed channels. Commonly used in concurrency models. Used in RPC, RMI, and MPI paradigms, Java RMI, CORBA, DDS, MSMQ, MailSlots, QNX, others
Memory-mapped file A file mapped to RAM and can be modified by changing memory addresses directly instead of outputting to a stream. This shares the same benefits as a standard file. All POSIX systems, Windows

Applications

Remote procedure call interfaces

Java's Remote Method Invocation (RMI)
ONC RPC
XML-RPC or SOAP
JSON-RPC
Message Bus (Mbus) (specified in RFC 3259)
.NET Remoting
gRPC

Platform communication stack

The following are messaging, and information systems that utilize IPC mechanisms but don't implement IPC themselves: KDE's Desktop Communications Protocol (DCOP) – deprecated by D-Bus D-Bus OpenWrt uses ubus micro bus architecture MCAPI Multicore Communications API SIMPL The Synchronous Interprocess Messaging Project for Linux (SIMPL) 9P (Plan 9 Filesystem Protocol) Distributed Computing Environment (DCE) Thrift ZeroC's Internet Communications Engine (ICE) ØMQ Enduro/X Middleware YAMI4

Operating system communication stack

The following are platform or programming language-specific APIs: Linux Transparent Inter Process Communication (TIPC) Apple Computer's Apple events, previously known as Interapplication Communications (IAC) Enea's LINX for Linux (open source) and various DSP and general-purpose processors under OSE The Mach kernel's Mach Ports Microsoft's ActiveX, Component Object Model (COM), Microsoft Transaction Server (COM+), Distributed Component Object Model (DCOM), Dynamic Data Exchange (DDE), Object Linking and Embedding (OLE), anonymous pipes, named pipes, Local Procedure Call, MailSlots, Message loop, MSRPC, .NET Remoting, and Windows Communication Foundation (WCF) Novell's SPX POSIX mmap, message queues, semaphores,[4] and shared memory RISC OS's messages Solaris Doors System V's message queues, semaphores, and shared memory OpenBinder Open binder QNX's PPS (Persistent Publish/Subscribe) service

Distributed object models

The following are platform or programming language specific-APIs that use IPC, but do not themselves implement it: Libt2n for C++ under Linux only, handles complex objects and exceptions PHP's sessions Distributed Ruby Common Object Request Broker Architecture (CORBA)

Transparent Inter Process Communication service ( TIPC )

Transparent Inter Process Communication (TIPC) is an Inter-process communication (IPC) service in Linux designed for cluster wide operation. It is sometimes presented as Cluster Domain Sockets, in contrast to the well-known Unix Domain Socket service; the latter working only on a single kernel.

anitsh commented 3 years ago

mmap

"In computing, mmap is a POSIX-compliant Unix system call that maps files or devices into memory. It is a method of memory-mapped file I/O. It implements demand paging because file contents are not read from disk directly and initially do not use physical RAM at all. The actual reads from disk are performed in a "lazy" manner, after a specific location is accessed. After the memory is no longer needed, it is important to munmap the pointers to it. Protection information can be managed using mprotect(2), and special treatment can be enforced using madvise.

In Linux, macOS and the BSDs, mmap can create several types of mappings. Other operating systems may only support a subset of these; for example, shared mappings may not be practical in an operating system without a global VFS or I/O cache. " - https://en.wikipedia.org/wiki/Mmap

"POSIX environments provide at least two ways of accessing files. There's the standard system calls open(), read(), write(), and friends, but there's also the option of using mmap() to map the file into virtual memory. When is it preferable to use one over the other? What're their individual advantages that merit including two interfaces?"