For @wrapper imports, String is not a valid return type of interface/abstract methods

[rahulmutt]

foreign import java unsafe "@wrapper render" mkResTransformer :: (a <: Object) => (a -> Java (ResponseTransformer a) String) -> ResponseTransformer a

This code gives a compiler panic when it shouldn't.

[pparkkin]

What should it do?

Looks like the issue is that String of course is [Char], and DsForeign.getPrimTyOf panics with any type of list.

What happens is DsForeign.getPrimTyOf calls splitDataProductType_maybe in compiler/ETA/BasicTypes/DataCon.hs, and splitDataProductType_maybe returns a Nothing for anything with more than one constructor.

[rahulmutt]

I wonder if it's worth supporting this use case because it's going to be a special case which I'm not too sure will be worth it. String's are inefficient to have as a serialisation type since they are linked list of Char's and there's quite a bit of overhead to convert them. Instead, we can just make a comment that for @wrapper imports and exports, String will not be supported.

Now you may say the same argument applies to deprecating support for String for normal foreign imports, but it's used quite frequently throughout base, so we'll at least keep that for now.

Thoughts on this?

[jneira]

Mmm, but it supposes some assymetry between normal and wrappers fi's and it usually causes minor frustations what would be the alternative, use JString?

[rahulmutt]

Sounds good, we'll go with that then. @pparkkin Here's the implementation plan:

• When a user puts a String in either export/wrapper imports, that should indicate that the export type is java.lang.String. You'll have to add a special case to getPrimTyOf.
• If the String is put in an input position to the wrapper argument function or a foreign export function, you'll need to convert a java.lang.String to an Eta String when calling into the corresponding Eta function. You can generate jvm bytecode to:
• call unpackCString# with the input java.lang.String (viewed as Object# JString in Eta) by creating an ApO thunk.
• If the String is in the return position, you must convert an Eta String to a java.lang.String. You can call toJString and retrieve the java.lang.String stored inside.

Code-wise you only have to do a single modification for each change since the code generation for wrapper imports and foreign exports share a similar core.

This could be a bit tricky because it requires some knowledge of internals - let me know if you need me to expand more on any of the points above.

[pparkkin]

Got it!

I'll get back to you with any questions I have.

[pparkkin]

Since it's been a while, I better update on what's up, and ask for some help since I'm starting to feel pretty stuck.

First thing I did was just make a simple interface that has a single method that returns a String to use as an example and a test.

public interface StringProvider {
  String getString();
}

Next, the first change, getPrimTyOf, was pretty simple.

But getting to the code generation I got pretty lost. I've been trying to understand the dumped STG, and tracing through compiler/ETA/CodeGen/Foreign.hs and compiler/ETA/CodeGen/Main.hs. The STG looks something like this. I feel at this point I have a light grasp of what's going on.

        let {
          sat_s2VF [Occ=Once]
            :: GHC.Types.Java Main.StringProvider [GHC.Types.Char]
          [LclId, Str=DmdType] =
              \u srt:SRT:[0K :-> GHC.CString.unpackCString#,
                          r89 :-> GHC.Base.$fMonadJava, r8C :-> GHC.Base.$fFunctorJava,
                          r8G :-> GHC.Base.$fApplicativeJava] []
                  let {
                    sat_s2VE [Occ=Once] :: GHC.Base.String
                    [LclId, Str=DmdType] =
                        \u srt:SRT:[0K :-> GHC.CString.unpackCString#] []
                            GHC.CString.unpackCString# "Provider"#; } in
                  let {
                    sat_s2VC [Occ=Once]
                      :: GHC.Base.Applicative (GHC.Types.Java Main.StringProvider)
                    [LclId, Str=DmdType] =
                        \u srt:SRT:[r8C :-> GHC.Base.$fFunctorJava,
                                    r8G :-> GHC.Base.$fApplicativeJava] []
                            GHC.Base.$fApplicativeJava GHC.Base.$fFunctorJava;
                  } in
                    case GHC.Base.$fMonadJava sat_s2VC of sat_s2VD {
                      __DEFAULT -> GHC.Base.return sat_s2VD sat_s2VE;
                    };
        } in
          case
              __pkg_ccall_value False|False|0,"eta/eta/test/basic/StringProvider$Eta","(Leta/runtime/stg/Closure;)V" [sat_s2VF]
          of
          sat_s2VG
          { __DEFAULT ->
                case
                    __pkg_ccall_value main False|True|2,False,False,"eta/test/basic/StringPrinter","printString","(Leta/test/basic/StringProvider;)V" [sat_s2VG
                                                                                                                                                       eta_s2V9]
                of
                _ [Occ=Dead]
                { (##) ds_s2VI [Occ=Once, OS=OneShot] ->
                      (#,#) [ds_s2VI GHC.Tuple.()];
                };
          };

I'm not sure whether I should be wrapping the String in a conversion, as in this one.

                    sat_s2VE [Occ=Once] :: GHC.Base.String
                    [LclId, Str=DmdType] =
                        \u srt:SRT:[0K :-> GHC.CString.unpackCString#] []
                            GHC.CString.unpackCString# "Provider"#;

Or if this is where it's actually being called, and I should be doing it there.

                    case GHC.Base.$fMonadJava sat_s2VC of sat_s2VD {
                      __DEFAULT -> GHC.Base.return sat_s2VD sat_s2VE;
                    };

Also, I have no idea how I would even do it, even if I knew where.

Any help on how to proceed would be greatly appreciated.

[rahulmutt]

Ok, let's take a look at a sample import and see what needs to be done.

foreign import java unsafe "@wrapper getString" mkStringProvider ::
  Java StringProvider String -> StringProvider

Wrapper imports work by generating an auxiliary implementation of the interface called [interface]$Eta which has a single field of type eta.runtime.stg.Closure - a potentially lazy Eta value which should have type corresponding to the first argument above, in this case, Java StringProvider String.

When a caller calls StringProvider$Eta.getString(), it should run the Java StringProvider String action stored in the field of the implementation and run it using the runJava runtime function, which requires you to pass in an instance of StringProvider and you have one - this!

runJava will return you a Closure with type of the result, in this case [Char]. But you need to return java.lang.String from getString() in order to make it valid. How do you do this? You can apply the toJString function to [Char] to get JString. You can then unbox the JString and make it work.

If x is a value of type Java StringProvider String, then you need to generate code that will call runJava on fmap toJString x and unbox the result to java.lang.String.

This is the high level overview. Most of the implementation of the process I mentioned above is in dsFExport - you just need to add the fmap toJString bit in the right place. If you need additional help in which parts to modify, I'd be happy to give you more info.

One more thing, since you're tweaking how the core is generated, I suggest you add -dcore-lint to eta-options: in your test project's .cabal file. It's a quick way of knowing whether the core you're generating is at least valid.

[pparkkin]

Looking at dsFExport and the generated byte code, I think I’m starting to understand what's going on here. Let me just run it by you to make sure I’m not getting anything wrong.

The interface method implementation gets wrapped into a bunch of thunks before getting evaluated by eta.runtime.Runtime.evalJava(). First it gets wrapped into an Ap1Upd, I assume just to get it into a thunk. Next it gets wrapped into an Ap2Upd thunk together with the closure that is returned from base.java.TopHandler.runJava(), which I think is just the Java monad that all the Java stuff runs in. What it ends up as is something like Ap2Upd(runJava, Ap1Upd(getString Eta implementation)). That gets then passed into evalJava() along with this, which returns something which is a ghc_prim.ghc.Types$ZCD ([Char] type thunk?). Instead of ghc_prim.ghc.Types$ZCD the method expects a ghc_prim.ghc.Types$ZMZND (java.lang.String type thunk?).

So what I need is something like Ap2Upd(toJString, Ap2Upd(runJava, Ap1Upd(getString Eta implementation))) to pass into evalJava(), correct?

Do I need to do this at the byte code generation level, or can I somehow just inject it into the Core and have it automatically be picked up by the byte code generation?

[pparkkin]

@rahulmutt Okay, I think I finally may have figured out what I should be doing here.

Here's what I'm playing around with.

         $ mkMethodDef className accessFlags methodName argFts resFt $
             loadThis
          <> new ap3Ft
          <> dup ap3Ft
          <> invokestatic (mkMethodRef "base/ghc/Base" "fmap" [] (Just closureType))
          <> invokestatic (mkMethodRef "base/java/String" "toJString" [] (Just closureType))
          <> new ap2Ft
          <> dup ap2Ft
          <> invokestatic (mkMethodRef runClass runClosure
                                       [] (Just closureType))
          <> new apFt
          <> dup apFt
          <> loadClosureRef
          <> boxedArgs
          <> invokespecial (mkMethodRef apClass "<init>" (replicate numApplied closureType) void)
          <> invokespecial (mkMethodRef ap2Class "<init>" [ closureType
                                                          , closureType] void)
          <> invokespecial (mkMethodRef ap3Class "<init>" [ closureType
                                                          , closureType
                                                          , closureType] void)
          -- TODO: Support java args > 5
          <> invokestatic (mkMethodRef rtsGroup evalMethod evalArgFts (ret closureType))

But it's giving me an error.

Exception in thread "main" eta.runtime.exception.EtaException: JException java.lang.ClassCastException: base.java.String$toJString cannot be cast to base.ghc.Base$DZCFunctorD
    at base.ghc.Base$fmap.enter(Unknown Source)
    at eta.runtime.apply.Function.applyPP(Function.java:135)
    at eta.runtime.thunk.Ap3Upd.thunkEnter(Ap3Upd.java:21)
    at eta.runtime.thunk.UpdatableThunk.enter(UpdatableThunk.java:18)
    at eta.runtime.stg.Closure.evaluate(Closure.java:24)
    at eta.runtime.stg.Closures$EvalJava.enter(Closures.java:128)
    at eta.runtime.stg.Capability.schedule(Capability.java:150)
    at eta.runtime.stg.Capability.scheduleClosure(Capability.java:97)
    at eta.runtime.Runtime.evalJava(Runtime.java:197)
    at eta.eta.test.basic.StringProvider$Eta.getString(Unknown Source)
    at eta.test.basic.StringPrinter.printString(StringPrinter.java:6)
    at main.Main$$Ls2VJsat.enter(Unknown Source)
    at eta.runtime.apply.Function.applyV(Function.java:16)
    at base.ghc.Base$thenIO1.enter(Unknown Source)
    at eta.runtime.apply.PAP.apply(PAP.java:31)
    at eta.runtime.apply.PAP.applyV(PAP.java:41)
    at eta.runtime.stg.Closures$EvalLazyIO.enter(Closures.java:96)
    at eta.runtime.stg.Capability.schedule(Capability.java:150)
    at eta.runtime.stg.Capability.scheduleClosure(Capability.java:97)
    at eta.runtime.Runtime.evalLazyIO(Runtime.java:189)
    at eta.runtime.Runtime.main(Runtime.java:182)
    at eta.main.main(Unknown Source)
Caused by: java.lang.ClassCastException: base.java.String$toJString cannot be cast to base.ghc.Base$DZCFunctorD

I tried it with two nested Ap2Upd thunks in place of the Ap3Upd, but that didn't work. I also tried switching the argument order around, but I couldn't figure out a way to get it to work.