Unity Embedded Packages issues with Plugins, Unity Libraries, and exernal UPM Packages

[TCROC]

Embedded packages are not correctly loading references to Unity Libraries, Plugins, or external UPM Packages.

Here is the example repo: https://github.com/TCROC/ConsuloBug

The working normal script looks like this as expected:

The broken embedded script looks like this:

OS: Ubuntu 18.04 Unity Version: 2019.4.14f1 Unity Consulo Plugin Version: 2.4.0

Consulo About

 Consulo:
  version = 2020.12
  build number = 2806
  build date = December 7, 2020
 Plugins:
  com.intellij = 2806
  com.intellij.xml = 2657
  com.intellij.properties = 2761
  consulo.dotnet = 3284
  consulo.dotnet.mono = 3833
  consulo.csharp = 4831
  consulo.javascript = 3066
  consulo.java = 4351
  com.intellij.uiDesigner = 5391
  consulo.nunit = 4363
  consulo.platform.desktop = 2806
  org.jetbrains.idea.maven = 5318
  org.jetbrains.plugins.yaml = 2127
  consulo.devkit = 8572
  consulo.unity3d = 7516
 JVM:
  vendor = JetBrains s.r.o
  version = 11.0.4
  runtimeVersion = 11.0.4+10-b304.77
  locale = en_US
 JVM Env:
  __idea.mac.env.lock = unlocked
  awt.toolkit = sun.awt.X11.XToolkit
  java.specification.version = 11
  sun.cpu.isalist = 
  sun.jnu.encoding = UTF-8
  log4j.shutdownHookEnabled = false
  sun.arch.data.model = 64
  consulo.properties.file = /home/t-croc/Applications/Consulo/consulo.properties
  sun.font.fontmanager = sun.awt.X11FontManager
  idea.properties.file = /home/t-croc/Applications/Consulo/consulo.properties
  java.vendor.url = https://www.jetbrains.com/
  consulo.home.path = /home/t-croc/Applications/Consulo/platform/build2806
  sun.java2d.uiScale.enabled = true
  log4j2.messageFactory = org.apache.logging.log4j.message.SimpleMessageFactory
  sun.io.useCanonCaches = false
  sun.boot.library.path = /home/t-croc/Applications/Consulo/platform/build2806/jre/lib
  jna.tmpdir = /home/t-croc/.consulo_settings/system/tmp
  sun.java.command = consulo.desktop.bootstrap/consulo.desktop.boot.main.Main --no-recent-projects --json /tmp/8f57893e-10df-4bd2-9f2f-bf43b64f832f.json
  jdk.debug = release
  sun.awt.exception.handler = com.intellij.openapi.application.impl.AWTExceptionHandler
  java.specification.vendor = Oracle Corporation
  jdk.module.path = /home/t-croc/Applications/Consulo/platform/build2806/boot
  java.version.date = 2019-07-16
  java.home = /home/t-croc/Applications/Consulo/platform/build2806/jre
  file.separator = /
  java.vm.compressedOopsMode = 32-bit
  line.separator = \n
  java.specification.name = Java Platform API Specification
  java.vm.specification.vendor = Oracle Corporation
  idea.home.path = /home/t-croc/Applications/Consulo/platform/build2806
  jdk.module.main.class = consulo.desktop.boot.main.Main
  awt.useSystemAAFontSettings = lcd
  sun.management.compiler = HotSpot 64-Bit Tiered Compilers
  jb.restart.code = 88
  java.runtime.version = 11.0.4+10-b304.77
  user.name = t-croc
  javax.swing.rebaseCssSizeMap = true
  sun.java2d.pmoffscreen = false
  sun.awt.noerasebackground = true
  file.encoding = UTF-8
  sun.java2d.noddraw = true
  jnidispatch.path = /home/t-croc/.consulo_settings/system/tmp/jna17038015385769039002.tmp
  idea.popup.weight = heavy
  jna.loaded = true
  java.io.tmpdir = /tmp
  java.version = 11.0.4
  java.vm.specification.name = Java Virtual Machine Specification
  java.awt.printerjob = sun.print.PSPrinterJob
  sun.os.patch.level = unknown
  consulo.vm.options.file = /home/t-croc/Applications/Consulo/consulo64.vmoptions
  java.library.path = /home/t-croc/Applications/Consulo/platform/build2806/bin:/tmp/.mount_UnityHSvyhgU/usr/lib::/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
  java.vendor = JetBrains s.r.o
  io.netty.processId = 62084
  sun.io.unicode.encoding = UnicodeLittle
  java.rmi.server.disableHttp = true
  io.netty.machineId = 28:f0:76:ff:fe:16:65:0e
  java.util.concurrent.ForkJoinPool.common.parallelism = 8
  sun.desktop = gnome
  java.class.path = 
  sun.awt.enableExtraMouseButtons = true
  java.vm.vendor = JetBrains s.r.o
  consulo.module.path.boot = true
  user.timezone = America/New_York
  jb.vmOptionsFile = /home/t-croc/Applications/Consulo/consulo64.vmoptions
  idea.no.launcher = false
  swing.bufferPerWindow = false
  os.name = Linux
  java.vm.specification.version = 11
  sun.java.launcher = SUN_STANDARD
  user.country = US
  sun.cpu.endian = little
  user.home = /home/t-croc
  user.language = en
  idea.cycle.buffer.size = 1024
  java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
  jdk.module.main = consulo.desktop.bootstrap
  idea.max.intellisense.filesize = 2500
  sun.java2d.d3d = false
  java.net.preferIPv4Stack = true
  path.separator = :
  idea.use.default.antialiasing.in.editor = false
  os.version = 5.4.0-56-generic
  jna.nosys = true
  java.runtime.name = OpenJDK Runtime Environment
  sun.nio.ch.bugLevel = 
  java.vm.name = OpenJDK 64-Bit Server VM
  jna.platform.library.path = /usr/lib/x86_64-linux-gnu:/lib/x86_64-linux-gnu:/lib64:/usr/lib:/lib:/usr/lib/i386-linux-gnu:/lib/i386-linux-gnu:/lib32:/usr/lib32
  java.vendor.url.bug = https://youtrack.jetbrains.com
  java.util.concurrent.ForkJoinPool.common.threadFactory = consulo.bootstrap.concurrent.IdeaForkJoinWorkerThreadFactory
  user.dir = /home/t-croc/Applications/Consulo
  os.arch = amd64
  io.netty.serviceThreadPrefix = Netty 
  idea.dynamic.classpath = false
  java.vm.info = mixed mode
  java.vm.version = 11.0.4+10-b304.77
  java.rmi.server.hostname = localhost
  sun.awt.datatransfer.timeout = 2000
  java.class.version = 55.0
 OS:
  name = Linux
  version = 5.4.0-56-generic
  arch = amd64
 Env:
  PATH = /tmp/.mount_UnityHSvyhgU:/tmp/.mount_UnityHSvyhgU/usr/sbin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/t-croc/.dotnet/tools:/usr/local/bin:/usr/local/sbin:/opt/local/bin:/opt/local/sbin
  XAUTHORITY = /run/user/1000/gdm/Xauthority
  XMODIFIERS = @im=ibus
  XDG_DATA_DIRS = /tmp/.mount_UnityHSvyhgU/usr/share/:./share/:/usr/share/gnome:/usr/local/share/:/usr/share/:/usr/share/ubuntu:/usr/local/share/:/usr/share/:/var/lib/snapd/desktop:/usr/share/gnome/:/usr/local/share/:/usr/share/
  GDMSESSION = ubuntu
  MANDATORY_PATH = /usr/share/gconf/ubuntu.mandatory.path
  TEXTDOMAINDIR = /usr/share/locale/
  GTK_IM_MODULE = ibus
  DBUS_SESSION_BUS_ADDRESS = unix:path=/run/user/1000/bus
  DEFAULTS_PATH = /usr/share/gconf/ubuntu.default.path
  XDG_CURRENT_DESKTOP = Unity
  APPIMAGE = /home/t-croc/Applications/UnityHub.AppImage
  SSH_AGENT_PID = 2229
  LD_LIBRARY_PATH = /home/t-croc/Applications/Consulo/platform/build2806/bin:/tmp/.mount_UnityHSvyhgU/usr/lib:
  QT4_IM_MODULE = xim
  SESSION_MANAGER = local/tcroc-Z390-UD:@/tmp/.ICE-unix/2133,unix/tcroc-Z390-UD:/tmp/.ICE-unix/2133
  USERNAME = t-croc
  LOGNAME = t-croc
  PWD = /home/t-croc/Applications/Consulo
  IM_CONFIG_PHASE = 2
  GJS_DEBUG_TOPICS = JS ERROR;JS LOG
  SHELL = /bin/bash
  CHROME_DESKTOP = UnityHub.desktop
  GIO_LAUNCHED_DESKTOP_FILE = /home/t-croc/.local/share/applications/alacarte-made-1.desktop
  GNOME_DESKTOP_SESSION_ID = this-is-deprecated
  GTK_MODULES = gail:atk-bridge
  CLUTTER_IM_MODULE = xim
  __GL_SYNC_TO_VBLANK = 0
  TEXTDOMAIN = im-config
  ALL_JVM_ARGS = --add-opens=java.desktop/sun.awt=consulo.container.impl --add-opens=java.desktop/sun.awt.image=consulo.container.impl --add-opens=java.desktop/sun.java2d=consulo.container.impl --add-opens=java.desktop/sun.font=consulo.container.impl --add-opens=java.desktop/java.awt=consulo.container.impl --add-opens=java.desktop/javax.swing=consulo.container.impl --add-opens=java.desktop/javax.swing.plaf.basic=consulo.container.impl --add-opens=java.base/java.lang=consulo.container.impl --add-opens=java.desktop/javax.swing.text.html=consulo.container.impl --add-opens=java.desktop/java.awt.peer=consulo.container.impl  -Xms512m -Xmx1500m -XX:ReservedCodeCacheSize=240m -ea -Dsun.io.useCanonCaches=false -Djava.net.preferIPv4Stack=true -XX:+UseCodeCacheFlushing -XX:+UseConcMarkSweepGC -XX:SoftRefLRUPolicyMSPerMB=50 -XX:+HeapDumpOnOutOfMemoryError -Dawt.useSystemAAFontSettings=lcd  -Djb.vmOptionsFile="/home/t-croc/Applications/Consulo/consulo64.vmoptions" -Dconsulo.vm.options.file="/home/t-croc/Applications/Consulo/consulo64.vmoptions" -p "/home/t-croc/Applications/Consulo/platform/build2806/boot" -Dconsulo.module.path.boot=true -Dconsulo.home.path="/home/t-croc/Applications/Consulo/platform/build2806" -Didea.home.path="/home/t-croc/Applications/Consulo/platform/build2806" -Didea.properties.file="/home/t-croc/Applications/Consulo/consulo.properties" -Dconsulo.properties.file="/home/t-croc/Applications/Consulo/consulo.properties"   
  APPDIR = /tmp/.mount_UnityHSvyhgU
  XDG_SESSION_DESKTOP = ubuntu
  CONSULO_HOME = /home/t-croc/Applications/Consulo/platform/build2806
  SHLVL = 0
  QT_IM_MODULE = xim
  XDG_CONFIG_DIRS = /etc/xdg/xdg-ubuntu:/etc/xdg
  LANG = en_US.UTF-8
  XDG_SESSION_ID = 3
  XDG_SESSION_TYPE = x11
  JDK = /home/t-croc/Applications/Consulo/platform/build2806/jre
  DISPLAY = :1
  ROOT_DIR = /home/t-croc/Applications/Consulo
  OWD = /home/t-croc
  MAIN_CLASS_NAME = -m consulo.desktop.bootstrap/consulo.desktop.boot.main.Main
  GDK_BACKEND = x11
  GPG_AGENT_INFO = /run/user/1000/gnupg/S.gpg-agent:0:1
  ARGV0 = /home/t-croc/Applications/UnityHub.AppImage
  DESKTOP_SESSION = ubuntu
  USER = t-croc
  XDG_MENU_PREFIX = gnome-
  GIO_LAUNCHED_DESKTOP_FILE_PID = 4084
  QT_ACCESSIBILITY = 1
  WINDOWPATH = 2
  GJS_DEBUG_OUTPUT = stderr
  SSH_AUTH_SOCK = /run/user/1000/keyring/ssh
  XDG_SEAT = seat0
  NO_AT_BRIDGE = 1
  GSETTINGS_SCHEMA_DIR = /tmp/.mount_UnityHSvyhgU/usr/share/glib-2.0/schemas:
  GNOME_SHELL_SESSION_MODE = ubuntu
  XDG_VTNR = 2
  XDG_RUNTIME_DIR = /run/user/1000
  HOME = /home/t-croc

[VISTALL]

Hello. Depends to Assemby Definition implementation.

[TCROC]

What does that mean? I'm pretty sure its a bug. Unity doesn't throw any errors, but consulo does.

[VISTALL]

Consulo not analyze asmdef. Packages do not have dependencies to Project code. Maybe simple fix - it will add Assemby-CSharp dependency to emb. packages

[TCROC]

That sounds like a good fix :)

[VISTALL]

That sounds like a good fix :)

But Assemby-Csharp(main library) depends to all packages. And now we see recursion in dependencies.

Need understand what Unity add to it. maybe just binaries.

[TCROC]

Ah I see what you are saying. Embedded packages normally only occur when we are doing development on the package(s). Because consulo doesn't have asmdef support, what if we just add Embedded packages to Assemby-Csharp? Would that be a better fix?

[VISTALL]

Emmbedded packages already added to Assemby-CSharp as dependency. In your example, Plugins directory is part main source code, but Package depends to it. Maybe unity just ignore package, and work as one scope. Need analyze project files what unity generate

[TCROC]

Yes I agree. I think Unity ignores that Package for Embedded packages and works as one scope. I agree with that.

[VISTALL]

I'm not sure. I tested with your project. Unity generate VS C# project with this settings for Package(not global scope)

     <Compile Include="Packages\com.consulo.bug\Scripts\BrokenEmbeddedPackageScript.cs" />
     <Compile Include="Packages\com.consulo.bug\Scripts\Testing.cs" />
     <None Include="Packages\com.consulo.bug\consulo.bug.asmdef" />

then Unity dependencies

and

 <Reference Include="Newtonsoft.Json">
 <HintPath>W:/_github.com/TCROC/ConsuloBug/Assets/Plugins/Newtonsoft.Json.dll</HintPath>
 </Reference>

but no code. need more testing . strange

[TCROC]

We will be able to do that once we add in asmdef support. That will be the correct way of doing it. But until then, the workaround is to work as one scope. At least that is my current understanding of the situation.

[VISTALL]

We will be able to do that once we add in asmdef support. That will be the correct way of doing it. But until then, the workaround is to work as one scope. At least that is my current understanding of the situation.

Tested. It's only create dependencies to binaries inside Main sources (to packages). Funny. I will try to fix what, asmdef impl not needed

[VISTALL]

Test new build

[TCROC]

This still has some issues:

The plugin dlls are loading in along with the System.Net.Http that you fixed in the other issue, but Unity.UI, TMPro, and other external libraries are not loading in.

Green Checkmark = working as expected Red X = not working as expected

[VISTALL]

Added project packages as dependencies to embedded packages. Few min and update ill be arrive

[TCROC]

Sounds good. Just let me know when to pull it down :)

[VISTALL]

build. Need jenkins notifier for github commits

[TCROC]

Yay! It appears to be working! :)

I'm going to test it out in our main project quick to verify and if it works I'll close the issue

[TCROC]

Ok I think it is resolved. I found this one small issue that is a rare:

In this case, PlayFabMirror depends on PlayFab. PlayFab does not offer their sdk as a UPM as can be seen here. So it has to be installed to the main project. Sometimes we want to build UPM packages to extend that. However, in order to reference that without asmdef support, would have the circular dependency. The only solution that I see to this one is to add support for asmdef files as is being tracked by this issue https://github.com/consulo/consulo-unity3d/issues/186. So for now, I think this is the best solution without asmdef files and this can be closed.

[TCROC]

Thanks for fixing it :)