Build a big set of libraries

[daviditen]

This list is taken from the Python standard library. These are the Python modules that I think would be useful in Chapel. This is an incomplete list - for example this list only includes one compression algorithm, but the Python standard library has several. It does not include Python libraries that are not in the standard library.

The format of this list is: Category

• moduleName - Description

Data Types

• calendar - General calendar-related functions
• collections - Container data types
• heapq - Heap queue

Compression

• zlib & gzip - gzip compression support
• tarfile - tar archive support

File Formats

• csv - read/write CSV files
• configparser - parse configuration files
• netrc - file format used by FTP

Cryptography

• hashlib - secure hashes
• hmac - hashing for message authentication
• secrets - cryptographically strong random numbers

OS services

• argparse & getopt - command line parsing
• logging - Tools for logging Logging
• getpass - password input functionsConcurrency
• subprocess - handling subprocesses Spawn module
• sched - event scheduler
• queue - synchronized queue

Interprocess Communication & Networking

• socket - low level networking interface
• ssl - secure sockets layer
• signal - signal handling
• mmap - memory mapped files

Internet Data

• email - handling of email messages
• json - JSON encoder/decoder

Markup

• html - html handling
• xml - xml handling

Internet Protocols

• urllib - URL handling
• http - HTTP support
• poplib/imaplib/imaplib/smtplib - email support
• ipaddress - IP address handling/manipulation

Multimedia

• wave - read/write WAV files
• ossaudiodev - access OSS audio devices
• colorsys - convert between different color systems

Graphical User Interfaces

• tkinter - interface to tcl/tk
• PyQt/PySide - Qt interface
• wxPython - wxWidgets interface

Profiling

• cProfile - code profiling based on lsprof

[ben-albrecht]

Some libraries not listed above that will be needed by the package manager effort:

• git2
• tempfile
• toml

[ben-albrecht]

Flagged as [easy] since porting/wrapping libraries would be a good way for new Chapel developers to start contributing to the project.

@daviditen aptly pointed out that we will need some guidelines to help external contributors follow the conventions of introducing a new library to Chapel, e.g. getting feedback on a mockup interface, then a subset of the implementation, etc. ...

[dqian96]

Hey, my name is David and I am currently a software engineering student. I would like to tackle this this issue, how should I get started? Thanks!

[daviditen]

Hi David,

Thanks for your interest! What areas are you most interested in working on? Since this issue was written we've had some good progress on Collections (DistributedBag, DistributedDeque) and Cryptography, but most of the rest are not spoken for.

A good way to start would be to mock up an interface of what the new module would look like with descriptions of what exposed functions/classes/records/etc. do.

For example, if you took the signal handling point, I could imagine a mockup that started like:

/* A module for raising and handling signals */
module SignalHandling {
  /* POSIX signals */
  enum signal { ... };

  /* Raise the signal `sig` */
  proc raise(sig: signal): void {}

  /* set up a signal handler for the signal `sig` */
  proc handleSignal(sig: signal, handlerFunction): void {}
}

From there you could open an issue asking for input on your design and start getting to work implementing it.

[ben-albrecht]

@dqian96 - here are some examples of recent interface proposals that @daviditen was mentioning:

• 6654

• 6812

If you are interested in starting with a ramp-up task, you might consider contributing to an existing module. Many of the next steps for existing module are documented in the format of "Improve X" where "X" is a module name. This issue query captures most of these issues.

[dqian96]

@ben-albrecht - thanks for pointing me to those useful threads.

I think that improving an existing module would be a great place to start. I am interested in implement timSort for the sort module. I see that it was attempted before, but it was not merged. I think this would be a good place to start, as there already exists information on the implementation process.

Thanks

[dqian96]

I submitted a PR for the tim sort implementation #7782 Please let me know if I'm on the right track! Thanks

[krishnadey30]

@ben-albrecht @daviditen I would like to take up some of these issues. I would like to know more about heapq. You meant priority queue using heap right?

[ben-albrecht]

@krishnadey30 - yes, here is a good reference: https://docs.python.org/3.7/library/heapq.html

[ben-albrecht]

json - JSON encoder/decoder

@daviditen - Don't we already have support for this in our IO module? Do you think we could benefit from an additional JSON module similar to the TOML module?

csv - read/write CSV files

Perhaps we should remove this from the list now, or at least link to another issue that is tracking the progress on our csv module.

[lydia-duncan]

iirc, our JSON support is only used in formatted IO. @mppf is also a good person to ask

[mppf]

We need to pull the JSON encoder/decoder out of formatted I/O and into some sort of JSON support object. See #7846. It's true that it's currently only available through formatted I/O.

[lydia-duncan]

Also, isn't subprocess already handled?

[daviditen]

csv - read/write CSV files

Perhaps we should remove this from the list now, or at least link to another issue that is tracking the progress on our csv module.

There's a prototype in the test system but no official CSV module. The prototype would need design buy-in and better handling for malformed files before it could become an official module.

[ben-albrecht]

configparser - parse configuration files

From what I understand, this parses a format similar to Microsoft INI. I'm not sure there's much value to supporting this over other common config formats like TOML, YAML, and JSON, and suggest we drop it from this list.

[dgarvit]

Can I work on gzip library for chapel?

[mppf]

@dgarvit - I don't know of any other effort on that, so feel free

[DmiitriyJarosh]

Some libraries not listed above that will be needed by the package manager effort:

• git2
• tempfile
• toml

Hello, my name is Dmitrii. I am 3d year student of Saint-Petersburg State University in Russia. Implementing git2 sounds really great for me. Is it still able to be taken as a project? Can it be a part of GSOC program? Can you recommend me something to start with this topic?

[ben-albrecht]

Hi @DmiitriyJarosh --

Is it still able to be taken as a project?

Yes.

Can it be a part of GSOC program?

I think so, if you can identify someone to mentor the project -- see https://chapel-lang.org/gsoc/faq.html for info on proposing an original project.

Can you recommend me something to start with this topic?

I would suggest creating a separate issue for implementing a git2 Chapel interface, where further discussion about design and implementation can take place. Check out these guidelines for opening a design issue: ContributorInfo.rst#design

You will want to familiarize yourself with Chapel, specifically Chapel/C interop, for this project.

[DmiitriyJarosh]

Hi @DmiitriyJarosh --

Is it still able to be taken as a project?

Yes.

Can it be a part of GSOC program?

I think so, if you can identify someone to mentor the project -- see https://chapel-lang.org/gsoc/faq.html for info on proposing an original project.

Can you recommend me something to start with this topic?

I would suggest creating a separate issue for implementing a git2 Chapel interface, where further discussion about design and implementation can take place. Check out these guidelines for opening a design issue: ContributorInfo.rst#design

You will want to familiarize yourself with Chapel, specifically Chapel/C interop, for this project.

I created issue #15143 for design discussion

[ben-albrecht]

@egeOzpinar - AFAIK, yes.

[egeOzpinar]

@ben-albrecht Okay then I'll start as soon as possible. Thank you.

[mukul-mehta]

Hi! Could I work on the logging module? I'd also like to work on the argparse module.

[ben-albrecht]

@mukul-mehta - there is actually a logging mason package: https://github.com/ajoshpratt/chpllog

You might be able to contribute to that package though.

[ben-albrecht]

I'd also like to work on the argparse module.

That would be great.

[Aniket21mathur]

@ben-albrecht I like to work on the gzip library, is it available?

[ben-albrecht]

@ben-albrecht I like to work on the gzip library, is it available?

AFAIK, yes. That would be great!

[parthsarthiprasad]

@ben-albrecht Is work on support for file formats available? I've worked on integrating jpeg support for chapel as personal project once.

[ben-albrecht]

@parthsarthiprasad - I think so. A standalone jpeg package would be nice.

[nimitbhardwaj]

Is the wave multimedia library implemented? I have done some work with sounds in my projects with python at the root level of samples, I may work on it if it is not implemented yet.

[ben-albrecht]

@nimitbhardwaj - No one has worked on that one to my knowledge.

[rapiz1]

For those who're interested in heapq, Heap is already in Chapel. Maybe heapq can be somehow extending Heap.

[ben-albrecht]

Removing [good first issue]. This mega-issue is useful for external contributors to find projects to work on in Chapel, but is not really a great starting point for a first-time contributor.

@daviditen - separately, I wonder if we should repackage this issue as "mason package ideas" since we don't expect any of these libraries to be contributed to the standard library initially.