extern implementaion

[longlongman]

Hi, in the v1model.p4 https://github.com/p4lang/p4c/blob/master/p4include/v1model.p4, I found that there are just some extern declarations. 1) I want to know where their implementations are if I use the software target simple_switch. 2) And can I define my own extern structure except those the v1model has offered to me? 3) I noticed that the packet_in, packet_out are also extern type, but they are built in p4 language. Are there any differences between these two and those in v1model?

[antoninbas]

1. bmv2 was written when v1model (which is the P4_16 architecture which implements the legacy P4_14 abstract switch model) was the de facto architecture for everyone and in a way all of v1model extern types (e.g. counters, meters), which were P4_14 built-in objects, are supported "natively" by bmv2. The v1model extern functions, along with legacy P4_14 primitive actions, are implemented here: https://github.com/p4lang/behavioral-model/blob/master/targets/simple_switch/primitives.cpp

2. Yes you can implement your own extern types in bmv2. In theory, adding a new extern type to an existing architecture means that you are no longer using the same architecture. So by adding a new extern type to v1model, you are no longer writing P4 programs for v1model, but for _yourv1model, and you cannot expect your program to be supported by targets which claim v1model support. bmv2 being a SW target, it does offer you the flexibility to add your own externs to v1model and still run your P4 program with simple_switch. You may be able to get some detailed information on this by doing a Google search since we have had multiple questions in the past regarding how to do this. I think all you need to do now is:

   • implement and register your extern type (including its methods) somewhere in the simple_switch source code. You can find an example here: https://github.com/p4lang/behavioral-model/blob/master/tests/test_extern.cpp#L33
   • add your extern type definition to v1model.p4
   • compile your P4 program as you normally would, but use the --emit-externs flag

3. These built-in externs are supported natively in bmv2. Note that they are not like regular externs, in the sense that they are not instantiated in the program. When generating the bmv2 JSON, the compiler translates method calls on these externs to the appropriate bmv2 statements used to serialize / deserialize packets.

[hesingh]

In a recent Slack channel for p4, it was asked about crypto extern development for public and private key crypto. I don't fully understand what code to change in the behavioral-model, but here are code changes to v1model.p4 and partial changes to behavioral-model.

Let's first agree if (a) we should add crypto API to v1model.p4, and then (b) agree about the API which is included below.

diff --git a/p4include/v1model.p4 b/p4include/v1model.p4
index 95cbf6b7..ecce72bc 100644
--- a/p4include/v1model.p4
+++ b/p4include/v1model.p4
@@ -627,6 +627,9 @@ extern void assume(in bool check);
 extern void log_msg(string msg);
 extern void log_msg<T>(string msg, in T data);

+extern int encrypt(string msg);
+extern decrypt(int c, int priv_key);
+
 // The name 'standard_metadata' is reserved

 /*

The behavioral-model partial changes are:

diff --git a/tests/test_extern.cpp b/tests/test_extern.cpp
index 8b81313..aff6ce8 100644
--- a/tests/test_extern.cpp
+++ b/tests/test_extern.cpp
@@ -27,6 +27,13 @@
 #include <utility>
 #include <vector>
 #include <cassert>
+#include <iostream>
+
+extern "C" {
+#include <math.h>
+#include <stdlib.h>
+}
+#define SIZE 100

 using namespace bm;

@@ -96,6 +103,81 @@ BM_REGISTER_EXTERN_W_NAME_METHOD(counter2, ExternCounter, get);
 constexpr unsigned int ExternCounter::PACKETS;
 constexpr unsigned int ExternCounter::BYTES;

+class ExternCrypto : public ExternType {
+private:
+  // Returns gcd of a and b
+  int gcd(int a, int h)
+  {
+    int temp;
+    while (1)
+         {
+        temp = a%h;
+        if (temp == 0)
+          return h;
+        a = h;
+        h = temp;
+         }
+  }
+
+public:
+  void init() {
+    // Two random prime numbers
+    double p = 3;
+    double q = 7;
+
+    // First part of public key:
+    n = p*q;
+
+    // Finding other part of public key.
+    // e stands for encrypt
+    e = 2;
+    double phi = (p-1)*(q-1);
+    while (e < phi)
+    {
+        // e must be co-prime to phi and
+        // smaller than phi.
+        if (gcd(e, phi)==1)
+            break;
+        else
+            e++;
+    }
+
+    // Private key (d stands for decrypt)
+    // choosing d such that it satisfies
+    // d*e = 1 + k * totient
+    int k = 2;  // A constant value
+    d = (1 + (k*phi))/e;
+  }
+
+  void encrypt(char* msg) {
+    // Encryption c = (msg ^ e) % n
+    double ret = strtod(msg, nullptr);
+    double c = pow(ret, e);
+    c = fmod(c, n);
+    std::cout << "\nEncrypted data = " << c << std::endl;
+  }
+
+  void decrypt(int c, int priv_key) {
+    // Decryption m = (c ^ d) % n
+    double m = pow(c, priv_key);
+    m = fmod(m, n);
+       char output[SIZE];
+    snprintf(output, SIZE, "%f", m);
+    std::cout << "\nOriginal Message Sent = " << output << std::endl;
+  }
+
+private:
+  double n; // public key
+  double e; // public key
+  double d; // private key
+};
+
+// TODO: Code does not pass 'make check'
+// BM_REGISTER_EXTERN(ExternCrypto);
+//BM_REGISTER_EXTERN_METHOD(ExternCrypto, init);
+//BM_REGISTER_EXTERN_METHOD(ExternCrypto, encrypt);
+//BM_REGISTER_EXTERN_METHOD(ExternCrypto, decrypt);
+
 // this demonstrates how p4objects can be used in an extern for more advanced
 // use cases
 class ExternP4Objects : public ExternType {

[jafingerhut]

Copy of a reply I made on P4 Slack:

I do not know if there is interest in adding extern functions to v1model for encryption, but I do think there is value in having an article explaining the steps to add such an extern to both p4c and simple_switch, in a public Github repo, with diff files that can be applied to the currently latest p4c and behavioral-model repo code, so that the occasional person who asks how to do this can learn from a working example.

Instructions on how to make simple extensions like that seem likely to help more people than giving them another built-in thing.

[hesingh]

Regarding need, not only is inter-data center traffic encrypted, even intra-data center trafifc is. I have done what I could. Now, anyone familiar with the behavioral-model should complete the code. On code completion, we will also have an example of how a new extern is added to simple_switch and also support for crypto.

[antoninbas]

If this is about adding a new extern to v1model.p4, I do not believe this is the best forum to discuss this. Furthermore, I don't think it is likely that new functionality will be added to v1model. The charter for the architecture was to provide feature-parity with the P4_14 abstract switch model. Pretty much all changes to v1model in the last couple years fall in the category of "bug fixes".

You can still bring this up at the LDWG, but I doubt there will be too much interest. I'm not sure something like this should even be part of a future release of PSA. Even if encryption is becoming ubiquitous in data centers, it is not supported by switching chips and I doubt it is handled at the ToR. There could be interest on the NIC side however. @jafingerhut I don't know if you are still having meetings for "PNA".

I agree with Andy that a complete guide on how to add a custom extern to v1model for the sake of experimentation, along with a running example, would be very welcome. I don't have time to tackle this myself at the moment, but I would be happy to review it. I recommend making it clear in the guide that by adding a new extern to v1model, you effectively define a new architecture - my-v1model. However, the bmv2 compiler backend will still be able to compile the program and generate a "correct" JSON that can be consumed by simple_switch: either by a modified simple_switch binary that includes an implementation for the extern, or by the original simple_switch binary and the use of the --load-modules flag (which enables loading a shared library into bmv2 with an implementation of the extern).

[hesingh]

No problem if v1model.p4 is not changed and a guide is published.

Routing (Cisco CPP) and NIC ascis already combine 100G crypto. A switch could use switching asic and Marvell Nitrox for crypto. The crypto is still configured by switching asic.

[antoninbas]

I don't know what you mean by "The crypto is still configured by switching asic."

I think I see your point however. The problem is that a function like this is unlikely to get incorporated into something like PSA IMO, which aims to be portable. If you need to combine a switching ASIC and a separate processor for encryption / decryption, I think you can either 1) describe a new architecture that will work for your specific hardware combination, or 2) use a PSA program for the switching ASIC and steer traffic to the security processor by sending it to the appropriate port.

[hesingh]

The switching asic is configured by a cpu on the switch. An ACL is configured in switching asic to encrypt/decrypt traffic to/from certain ports and send/receive traffic to/from the Nitrox (crypto engine) bus.

If 2) is used, then no crypto externs are needed. If 1) is used, the data plane includes a crypto engine and crypto externs and the new architecture is needed.

[hesingh]

RSA asymmetric crypto is best implemented using gmp because gmp includes mpz_powm. However, the behavioral-model uses Bignum. If I use GMP, I will have to add gmp support to https://github.com/p4lang/behavioral-model/blob/master/include/bm/bm_sim/data.h

I have a repo that has developed RSA crypto and tested for encrypt and decrypt. It should be quick to add this C code as C++ code into tests_extern.cpp and complete API for encrypt and decrypt. But first, I'd have to add gmp support to data.h or develop power and modulus combined operation in Bignum.

https://github.com/hesingh/rsa

[antoninbas]

Look at https://github.com/p4lang/behavioral-model/blob/master/include/bm/bm_sim/bignum.h, you can get the mpz_t object by calling backend() on the Bignum. Your code would break if we ever change the Bignum implementation, but that's unlikely.

[hesingh]

I think we can close this issue now that Sa Pham in the P4 Slack channel found an implementation of an extern with simple_switch. See

https://github.com/uni-tue-kn/p4-macsec

1. The extern is defined here on the line below. The extern is also registered in the same file. https://github.com/uni-tue-kn/p4-macsec/blob/master/p4/target/simple_switch.cpp#L93

2. Then see basic.p4 which has declared the extern and used it in the following places.

https://github.com/uni-tue-kn/p4-macsec/blob/master/p4/p4/basic.p4#L178 https://github.com/uni-tue-kn/p4-macsec/blob/master/p4/p4/basic.p4#L281 https://github.com/uni-tue-kn/p4-macsec/blob/master/p4/p4/basic.p4#L398

3. The simple_switch.cpp file also implements encrypt and decrypt for packets.

I think the code is a good example to learn instructions on how to develop a new extern with simple_switch.

Another question:

Since the implementation has not changed v1model.p4, would the community consider incorporating the code in behavioral-model and simple_switch? Then we could issue a PR and review the code.

[antoninbas]

It's a new extern type and therefore technically a new architecture, so I'd rather not have non-v1model extern types enabled by default in simple_switch.

What we could do is place some "useful" externs in a separate library and document how to load it at runtime into bmv2 using --load-modules. Alternatively we can have a new command-line flag that just "enables" these extern types when explicitly required. We should also prefix all the names with bm_ or sswitch_ to make it obvious that these are not "standard".

I feel like at the moment a good place for something like this would be under targets/simple_switch/externs/extra/. What I don't know is what would be the criteria for inclusion in that semi-standard extern library. Maybe we should limit ourselves to 2 or 3 extern types and name the directory targets/simple_switch/externs/example/. On this, I tend to agree with @jafingerhut:

Instructions on how to make simple extensions like that seem likely to help more people than giving them another built-in thing.

Having one "crypto" extern in that list of extern examples sounds like the right thing to do.

[hesingh]

Appreciate the expedited reply.

I like targets/simple_switch/externs/crypto and also 2-3 extern types per example directory. Sometimes, even one extern such as crypto is good for one example.

However, I'd also like sample P4 code using the extern to also be added someplace so that P4 newbies are helped. Maybe, we can strick the P4 code in the same example directory.

Also, notice that the macsec repo has mirrored the behavioral-model p4/target directory and changed code.

https://github.com/uni-tue-kn/p4-macsec/tree/master/p4/target

[hesingh]

Do we need any more discussion such as in a LDWG meeting or is there consensus to create a new directory called externs under targets/simple_switch and let's do so. Later, one can add the crypto extern in the new path.

[jafingerhut]

One of the motivations for P4_16 was specifically that externs could be defined and added to an architecture, without changing the language definition. Adding example externs to behavioral-model, that do not change the current definition of the v1model architecture, but new README instructions state clearly how to compile and run a modified version of simple_switch or simple_switch_grpc with the new example externs enabled, on top of what v1model provides, impacts neither the P4_16 language, nor the current v1model definition.

[hesingh]

@jafingerhut thanks for your input. I think we could create a PR to add the new directory to the behavioral-model - this is minor work. Later we can add a crypto extern example under the new directory with another PR.

[jafingerhut]

I don't see much value in having a PR that simply creates the directory, with no examples in it. Why not simply create the directory when there is a first working example?

[hesingh]

Ok, sounds good.

[hesingh]

A PR has been issued. Please see

https://github.com/p4lang/behavioral-model/pull/834

[hesingh]

1. bmv2 was written when v1model (which is the P4_16 architecture which implements the legacy P4_14 abstract switch model) was the de facto architecture for everyone and in a way all of v1model extern types (e.g. counters, meters), which were P4_14 built-in objects, are supported "natively" by bmv2. The v1model extern functions, along with legacy P4_14 primitive actions, are implemented here: https://github.com/p4lang/behavioral-model/blob/master/targets/simple_switch/primitives.cpp
2. Yes you can implement your own extern types in bmv2. In theory, adding a new extern type to an existing architecture means that you are no longer using the same architecture. So by adding a new extern type to v1model, you are no longer writing P4 programs for v1model, but for _yourv1model, and you cannot expect your program to be supported by targets which claim v1model support. bmv2 being a SW target, it does offer you the flexibility to add your own externs to v1model and still run your P4 program with simple_switch. You may be able to get some detailed information on this by doing a Google search since we have had multiple questions in the past regarding how to do this. I think all you need to do now is:

• implement and register your extern type (including its methods) somewhere in the simple_switch source code. You can find an example here: https://github.com/p4lang/behavioral-model/blob/master/tests/test_extern.cpp#L33
• add your extern type definition to v1model.p4
• compile your P4 program as you normally would, but use the --emit-externs flag

1. These built-in externs are supported natively in bmv2. Note that they are not like regular externs, in the sense that they are not instantiated in the program. When generating the bmv2 JSON, the compiler translates method calls on these externs to the appropriate bmv2 statements used to serialize / deserialize packets.

A simple example of extern is available in the following path:

https://github.com/p4lang/behavioral-model/tree/master/targets/psa_switch/externs/psa_switch.h[.cpp]

Thereafter, incorporate the new extern .cpp file into appropriate Makefile.xx. See below:

https://github.com/p4lang/behavioral-model/blob/884e01b531c6fd078cc2438a40258ecae011a65b/targets/psa_switch/Makefile.am#L11

[hesingh]

I think the notes I added today suffice to guide new folks for how to add a new extern to simple_switch. Let's try to close this issue.

[jafingerhut]

@antoninbas No pressure, but do you perhaps know what commands one can run to compile the proposed new .cpp file in the PR on an Ubuntu 16.04 Linux system, and then what command line options should be added to a simple_switch/simple_switch_grpc command line, to make these new externs work? I'd be happy to test them and add a README to this example code given such commands, I'm just not sure I want to experiment and dig to find those commands.

[jafingerhut]

@hesingh I think the goal should be to leave behind something that we can point people to when they ask how to add externs, preferably with as many step-by-step instructions as we can provide. Otherwise, they are just going to ask again (yes, I know some will ask again even with the step-by-step instructions, but at least it will not be because we didn't write them down).

[hesingh]

@jafingerhut Ok, understood. Here is a start at step-by-step instructions.

1. Define new extern in a .h file. For example, see https://github.com/p4lang/behavioral-model/blob/master/targets/psa_switch/externs/psa_counter.h

2. Implement extern and register extern in .cpp. For example, see https://github.com/p4lang/behavioral-model/blob/master/targets/psa_switch/externs/psa_counter.cpp

3. Add building of .cpp to Makefile.am. For example, see https://github.com/p4lang/behavioral-model/blob/884e01b531c6fd078cc2438a40258ecae011a65b/targets/psa_switch/Makefile.am#L11

4. From behavioral-model root directory, type ./configure to update Makefile.

5. How to add new CLI for new extern. For example, see https://github.com/p4lang/behavioral-model/commit/884e01b531c6fd078cc2438a40258ecae011a65b#diff-22a3dbe651c84fadee7af7d9187cbfa2R1910

6. Add import statement.

https://github.com/p4lang/behavioral-model/commit/884e01b531c6fd078cc2438a40258ecae011a65b#diff-5ae8a8504eb56bc8167d4b69469ed7e8R66

7. Implement methods in psa_switch.h[.cpp]. See

https://github.com/p4lang/behavioral-model/commit/884e01b531c6fd078cc2438a40258ecae011a65b#diff-73004d69b350a529df2cf8a92fcd13d9L34

8. From behavioral-model root directory, type make -j3 to build new extern for psa_switch.

Also see, this checkin to the behavioral-model repo that shows code changes for the example given above.

https://github.com/p4lang/behavioral-model/commit/884e01b531c6fd078cc2438a40258ecae011a65b

[hesingh]

I have completed editing the step-by-step instructions. Please review - thanks.

[hesingh]

Tactically, I think the step-by-step instructions, which are complete, could be added to README for behavioral-model.

[jafingerhut]

Wherever the step-by-step instructions are added, I think they should be not for PSA, but for this crypto example that is part of this PR, added to simple_switch. I would want to show people a fully functional example, preferably -- the PSA implementation is not that yet.

[hesingh]

Ok - thanks.

Since the authors of macsec have not replied to the Issue, I have abandoned porting their symmetric crypto code to the PR. I would have to spend lot of time with that code, especially since the Slack channel first asked for asymmetric crypto.

Now, I am writing my asymmetric crypto code for the PR. This would be a complete example for extern with simple_switch.

[jafingerhut]

Note: I don't think there is any need for a new extern to do something useful or correct according to some standard somewhere. Whatever the current crypto code happens to do is probably good enough for an example. One of the points of an example is just to show how to make something work, even if it is not exactly what any one person wants to add as a new extern.

I'd be happy with an example extern that had at least one in parameter, at least one out parameter, and did nothing more than add 7 to the in value and assign that to the out value. If I know then what commands to use to build the modified simple_switch (or run simple_switch with the linking to a .so shared library), and how to compile a P4_16 program that calls that extern, then it is up to the reader to replace that definition with something they actually want.