################## JPype User Guide ################## .. toctree:: :maxdepth: 2 JPype Introduction ****************** JPype is a Python module to provide full access to Java from within Python. Unlike Jython, JPype does not achive this by re-implementing Python, but instead by interfacing both virtual machines at the native level. This shared memory based approach achieves good computing performance, while providing the access to the entirety of CPython and Java libraries. This approach allows direct memory access between the two machines, implementation of Java interfaces in Python, and even use of Java threading. JPype Use Cases =============== Here are three typical reasons to use JPype. - Access to a Java library from a Python program (Python oriented) - Visualization of Java data structures (Java oriented) - Interactive Java and Python development including scientific and mathematical programming. Let's explore each of these options. Case 1: Access to a Java library -------------------------------- Suppose you are a hard core Python programmer. You can easily use lambdas, threading, dictionary hacking, monkey patching, been there, done that. You are hard at work on your latest project but you just need to pip in the database driver for your customers database and you can call it a night. Unfortunately, it appears that your customers database will not connect to the Python database API. The whole thing is custom and the customer isn't going to supply you with a Python version. They did send you a Java driver for the database but fat lot of good that will do for you. Stumbling through the internet you find a module that says it can natively load Java packages as Python modules. Well, it's worth a shot... So first thing the guide says is that you need to install Java and set up a ``JAVA_HOME`` environment variable pointing to the JRE. Then start the JVM with classpath pointed to customers jar file. The customer sent over an example in Java so you just have to port it into Python. .. code-block:: java package com.paying.customer; import com.paying.customer.DataBase public class MyExample { public void main(String[] args) { Database db = new Database("our_records"); try (DatabaseConnection c = db.connect()) { c.runQuery(); while (c.hasRecords()) { Record record = db.nextRecord(); ... } } } } It does not look too horrible to translate. You just need to look past all those pointless type declarations and meaningless braces. Once you do, you can glue this into Python and get back to what you really love, like performing dictionary comprehensions on multiple keys. You glance over the JPype quick start guide. It has a few useful patterns... set the class path, start the JVM, remove all the type declarations, and you are done. .. code-block:: python # Boiler plate stuff to start the module import jpype import jpype.imports from jpype.types import * # Launch the JVM jpype.startJVM(classpath=['jars/database.jar']) # import the Java modules from com.paying.customer import DataBase # Copy in the patterns from the guide to replace the example code db = Database("our_records") with db.connect() as c: c.runQuery() while c.hasRecords(): record = db.nextRecord() ... Launch it in the interactive window. You can get back to programming in Python once you get a good night sleep. Case 2: Visualization of Java structures ---------------------------------------- Suppose you are a hard core Java programmer. Weakly typed languages are for wimps, if it isn't garbage collected it is garbage. Unfortunately your latest project has suffered a nasty data structure problem in one of the threads. You managed to capture the data structure in a serialized form but if you could just make graph and call a few functions this would be so much easier. But the interactive Java shell that you are using doesn't really have much in the way of visualization and your don't have time to write a whole graphing applet just to display this dataset. So poking around on the internet you find that Python has exactly the visualization that you need for the problem, but it only runs in CPython. So in order to visualize the structure, you need to get it into Python, extract the data structures and, send it to the plotting routine. You install conda, follow the install instructions to connect to conda-forge, pull JPype1, and launch the first Python interactive environment that appear to produce a plot. You get the shell open and paste in the boilerplate start commands, and load in your serialized object. .. code-block:: python import jpype import jpype.imports jpype.startJVM(classpath = ['jars/*', 'test/classes']) from java.nio.file import Files, Paths from java.io import ObjectInputStream with Files.newInputStream(Paths.get("myobject.ser")) as stream: ois = ObjectInputStream(stream) obj = ois.readObject() print(obj) # prints org.bigstuff.MyObject@7382f612 It appears that the structure is loaded. The problematic structure requires you call the getData method with the correct index. .. code-block:: python d = obj.getData(1) > TypeError: No matching overloads found for org.bigstuff.MyObject.getData(int), > options are: public double[] org.bigstuff.MyObject.getData(double) public double[] org.bigstuff.MyObject.getData(int) Looks like you are going to have to pick the right overload as it can't figure out which overload to use. Darn weakly typed language, how to get the right type in so that you can plot the right data. It says that you can use the casting operators. .. code-block:: python from jpype.types import * d = obj.getData(JInt(1)) print(type(d)) # prints double[] Great. Now you just need to figure out how to convert from a Java array into our something our visualization code can deal with. As nothing indicates that you need to convert the array, you just copy out of the visualization tool example and watch what happens. .. code-block:: python import matplotlib.pyplot as plt plt.plot(d) plt.show() A graph appears on the screen. Meaning that NumPy has not issue dealing with Java arrays. It looks like ever 4th element in the array is zero. It must be the PR the new guy put in. And off you go back to the wonderful world of Java back to the safety of curly braces and semicolons. Case 3: Interactive Java ------------------------ Suppose you are a laboratory intern running experiments at Hawkins National Laboratory. (For the purpose of this exercise we will ignore the fact that Hawkins was shut down in 1984 and Java was created in 1995). You have the test subject strapped in and you just need to start the experiment. So you pull up Jupyter notebook your boss gave you and run through the cells. You need to added some heart wave monitor to the list of graphed results. The relevant section of the API for the Experiment appears to be .. code-block:: java package gov.hnl.experiment; public interface Monitor { public void onMeasurement(Measurement measurement); } public interface Measurement { public double getTime(); public double getHeartRate(); public double getBrainActivity(); public double getDrugFlowRate(); public boolean isNoseBleeding(); } public class Experiment { public void addCondition(Instant t, Condition c); public void addMoniter(Monitor m); public void run(); } The notebook already has all the test conditions for the experiment set up and the JVM is started, so you just need to implement the monitor. Based on the previous examples, you start by defining a monitor class .. code-block:: python from jpype import JImplements, JOverride from gov.hnl.experiment import Monitor @JImplements(Monitor) class HeartMonitor: def __init__(self): self.readings = [] @JOverride def onMeasurement(self, measurement): self.readings.append([measurement.getTime(), measurement.getHeartRate()]) def getResults(self): return np.array(self.readings) There is a bit to unpack here. You have implemented a Java class from within Python. The Java implementation is simply an ordinary Python class which has be decorated with ``@JImplements`` and ``@JOverride``. When you forgot to place the ``@JOverride``, it gave you the response:: NotImplementedError: Interface 'gov.hnl.experiment.Monitor' requires method 'onMeasurement' to be implemented. But once you added the ``@JOverride``, it worked properly. The subject appears to be getting impatient so you hurry up and set up a short run to make sure it is working. .. code-block:: python hm = HeartMonitor() experiment.addMonitor(hm) experiment.run() readings = hm.getResults() plt.plot(readings[:,0], readings[:,1) plt.show() To your surprise, it says unable to find method addMonitor with an error message:: AttributeError: 'gov.hnl.experiment.Experiment' object has no attribute 'addMonitor' You open the cell and type ``experiment.add``. The line completes with ``experiment.addMoniter``. Whoops, looks like there is typo in the interface. You make a quick correction and see a nice plot of the last 30 seconds pop up in a window. Job well done, so you set the runtime back to one hour. Looks like you still have time to make the intern woodlands hike and forest picnic. Though you wonder if maybe next year you should sign up for another laboratory. Maybe next year, you will try to sign up for those orbital lasers the President was talking about in the March. That sounds like real fun. (This advanced demonstration utilized the concept of Proxies_ and `Code completion`_) The JPype Philosophy ==================== JPype is designed to allow the user to exercise Java as fluidly as possible from within Python. We can break this down into a few specific design goals. - Make Java appear Pythonic. Make it so a Python programmer feels comfortable making use of Java concepts. This means making use of Python concepts to create very Python looking code and at times bending Python concepts to conform to Java's expectations. - Make Python appear like Java. Present concepts from Java with a syntax that resembles Java so that Java users can work with Python without a huge learning curve. - Present everything that Java has to offer to Python. Every library, package, and Java feature if possible should be accessible. The goal of bridge is to open up places and not to restrict flow. - Keep the design as simple as possible. Mixing languages is already complex enough so don't required the user to learn a huge arsenal of unique methods. Instead keep it simple with well defined rules and reuse these concepts. For example, all array types originate from JArray, and thus using one can also use isinstance to check if a class is an array type. Rather than introducing factory that does a similar job to an existing one, instead use a keyword argument on the current factory. - Favor clarity over performance. This doesn't mean not trying to optimize paths, but just as premature optimization is the bane of programmers, requiring writing to maximize speed is a poor long term choice, especially in a language such as Python where weak typing can promote bit rot. - If a new method has to be introduced, make it look familiar. Java programmers look to a method named "of" to convert to a type on factories such as a Stream, thus ``JArray.of`` converts a Python NumPy array to Java. Python programmers expect that memory backed objects can be converted into bytes for rapid transfer using a memory view, thus ``memoryview(array)`` will perform that task. - Provide an obvious way for both Python and Java programmers to perform tasks. On this front JPype and Python disagree. In Python's philosophy there should be one -- and preferably only one -- obvious way to do things. But we are bridging two worlds and thus obviousness is in the eye of the beholder. The end result is that JPype has a small footprint while providing access to Java (and other JVM based languages) with a minimum of effort. Languages other than Java ========================= JPype is primarily focused on providing the best possible wrapper for Java in Python. However, the Java Virtual Machine (JVM) is used for many popular languages such a Kotlin and Scala. As such JPype can be used for any language which used the JVM. That said each language has its own special properties that tend to be represented in different ways. If you would like JPype fully to operate on your particular language the following is required. - Set up a test bench for your language under the test directory. Use ivy to pull in the required jar files required to run it and exercise each of the required language features that need to be exercised. - Write a language specific quick start guide for your language defining how things should appear in both your language of choice and within Python highlighting those things that are different from how Java. - Set up a test harness that exercises your language for each language feature and place a setup script like ``test_java`` that builds the harness. Alternatives ============ JPype is not the only Python module of its kind that acts as a bridge to Java. Depending on your programming requirements, one of the alternatives may be a better fit. Specifically JPype is designed for clarity and high levels of integration between the Python and Java virtual machine. As such it makes use of JNI and thus inherits all of the benefits and limitations that JNI imposes. With JPype, both virtual machines are running in the same process and are sharing the same memory space and threads. JPype can thus intermingle Python and Java threads and exchange memory quickly. But by extension you can't start and stop the JVM machine but instead must keep both machines throughout the lifespan of the program. High integration means tightly coupled and thus it embodies the musketeers motto. If Python crashes, so does Java as they only have one process to live in. A few alternatives with different philosophies and limitations are given in the following section. Please take my review comments with the appropriate grain of salt. When I was tasked with finding a replacement for Matlab Java integration for our project test bench, I evaluated a number of alternatives Python bridge codes. I selected JPype primarily because it presented the most integrated API and documentation which would be suitable for getting physicists up to speed quickly. Thus your criteria may yield a different selection. JPype's underlying technology was underwhelming so I have had the pleasure of many hours reworking stuff under the hood. For more details on what you can't do with JPype, please see Limitations_. `Jython `_ ------------------------------- Jython is a reimplementation of Python in Java. As a result it has much lower costs to share data structures between Java and Python and potentially much higher level of integration. Noted downsides of Jython are that it has lagged well behind the state of the art in Python; it has a limited selection of modules that can be used; and the Python object thrashing is not particularly well fit in Java virtual machine leading to some known performance issues. `Py4J `_ --------------------------- Py4J uses a remote tunnel to operate the JVM. This has the advantage that the remote JVM does not share the same memory space and multiple JVMs can be controlled. It provides a fairly general API, but the overall integration to Python is as one would expect when operating a remote channel operating more like an RPC front-end. It seems well documented and capable. Although I haven't done benchmarking, a remote access JVM will have a transfer penalty when moving data. `Jep `_ ------------------------------------- Jep stands for Java embedded Python. It is a mirror image of JPype. Rather that focusing on accessing Java from within Python, this project is geared towards allowing Java to access Python as sub-interpreter. The syntax for accessing Java resources from within the embedded Python is quite similar with support for imports. Notable downsides are that although Python supports multiple interpreters many Python modules do not, thus some of the advantages of the use of Python many be hard to realize. In addition, the documentation is a bit underwhelming thus it is difficult to see how capable it is from the limited examples. `PyJnius `_ -------------------------------------------- PyJnius is another Python to Java only bridge. Syntax is somewhat similar to JPype in that classes can be loaded in and then have mostly Java native syntax. Like JPype, it provides an ability to customize Java classes so that they appear more like native classes. PyJnius seems to be focused on Android. It is written using Cython .pxi files for speed. It does not include a method to represent primitive arrays, thus Python list must be converted whenever an array needs to be passed as an argument or a return. This seems pretty prohibitive for scientific code. PyJnius appears is still in active development. `Javabridge `_ ------------------------------------------------------------------ Javabridge is direct low level JNI control from Python. The integration level is quite low on this, but it does serve the purpose of providing the JNI API to Python rather than attempting to wrap Java in a Python skin. The downside being of course you would really have to know a lot of JNI to make effective use of it. `jpy `_ ------------------------------------- This is the most similar package to JPype in terms of project goals. They have achieved more capabilities in terms of a Java from Python than JPype which does not support any reverse capabilities. It is currently unclear if this project is still active as the most recent release is dated 2014. The integration level with Python is fairly low currently though what they do provide is a similar API to JPype. `JCC `_ ------------------------------------------------ JCC is a C++ code generator that produces a C++ object interface wrapping a Java library via Java's Native Interface (JNI). JCC also generates C++ wrappers that conform to Python's C type system making the instances of Java classes directly available to a Python interpreter. This may be handy if your goal is not to make use of all of Java but rather have a specific library exposed to Python. `VOC _` ---------------------------------------------------------- A transpiler that converts Python bytecode into Java bytecode part of the BeeWare project. This may be useful if getting a smallish piece of Python code hooked into Java. It currently list itself as early development. This is more in the reverse direction as its goals are making Python code available in Java rather providing interaction between the two. `p2j `_ ---------------------------------------------- This lists itself as "A (restricted) python to java source translator". Appears to try to convert Python code into Java. Has not been actively maintained since 2013. Like VOC this is primilarly for code translation rather that bridging. About this guide ================ The JPype User Guide is targeted toward programmers who are strong in either Python who wish to make use of Java or those who are strong with Java and are looking to use Python as a Java development tool. As such we will compare and contrast the differences between the languages and provide examples suitable to help illustrate how to translate from one language to the other on the assumption that being strong in one language will allow you to easily grasp the corresponding relations in the other. If you don't have a strong background in either language an appropriate language tutorial may be necessary. JPype will hide virtually all of the JNI layer such that there is no direct access to JNI concepts. As such attempting to use JNI knowledge will likely lead to incorrect assumptions such as incorrectly attempting to use JNI naming and method signatures in the JPype API. Where JNI limitations do appear we will discuss the consequences imposed in programming. No knowledge of JNI is required to use this guide or JPype. JPype only works with Python 3, thus all examples will be using Python version 3 syntax and assume the use of the Python 3 new style object model. The naming conventions of JPype follow the Java rules rather than those of Python. This is a deliberate choice as it would be dangerous to try to mangle Java method and field names into Python conventions and risk a name collision. Thus if method must have Java conventions then the rest of the module should follow the same pattern for consistency. Getting JPype started --------------------- This document holds numerous JPype examples. For the purposes of clarity the module is assumed to have been started with the following command .. code-block:: python # Import the module import jpype # Allow Java modules to be imported import jpype.imports # Import all standard Java types into the global scope from jpype.types import * # Import each of the decorators into the global scope from jpype import JImplements, JOverride, JImplementationFor # Start JVM with Java types on return jpype.startJVM(convertStrings=False) # Import default Java packages import java.lang import java.util This is not the only style used by JPype users. Some people feel it is best to limit the number for symbols in the global scope and instead start with a minimalistic approach. .. code-block:: python import jpype as jp # Import the module jp.startJVM(convertStrings=False) # Start the module Either style is usable and we do not wish to force any particular style on the user. But as the extra ``jp.`` tends to just clutter up the space and implies that JPype should always be used as a namespace due to namespace conflicts, we have favored the global import style. JPype only exposes 40 symbols total including a few deprecated functions and classes. The 13 most commonly used Java types are wrapped in a special module ``jpype.types`` which can be used to import all for the needed factories and types with a single command without worrying about importing potentially problematic symbols. We will detail the starting process more later in the guide. See `Starting the JVM`_. JPype Concepts *************** At its heart, JPype is about providing a bridge to use Java within Python. Depending on your perspective that can either be a means of accessing Java libraries from within Python or a way to use Java using Python syntax for interactivity and visualization. This mean not only exposing a limited API but instead trying to provide the entirety of the Java language with Python. To do this, JPype maps each of the Java concepts to the nearest concept in Python wherever they are similar enough to operate without confusion. We have tried to keep this as Pythonic as possible, though it is never without some rough edges. Python and Java share many of the same concepts. Types, class, objects, function, methods, and members. But in other places they are rather different. Python lacks casting, type declarations, overloading, and many other features of a strongly typed language, thus we must expose those concepts into the Python syntax as best we can. Java for instance has class annotation and Python have class decorators. Both serve the purpose of augmenting a class with further information, but are very different in execution. We have broken the mapping down in nine distinct concepts. Some elements serve multiple functions. Type Factories These are meta classes that allow one to declare a particular Java type in Python. The result of type factories are wrapper classes. (JClass_ and JArray_) Factories also exist to implement Java classes from within Python (JProxy_) Meta Classes These are classes to describe different properties of Java classes such as to check if a class is an Interface. (JInterface_) Base Classes These are JPype names for Java classes in Python that exist without importing any specific Java class. Concepts such as Object, String, and Exception are defined and can be used in instance checks. For example, to catch all Java exceptions regardless of type, we would catch ``JException``. These are mainly for convenience though they do have some extra functionality. Most of these functions are being phased out in favor of Java syntax. For example, catching ``java.lang.Throwable`` will catch everything that ``JException`` will catch. (Jarray_, JObject_, JString_, and JException_) Wrapper Classes These correspond to each Java class. Thus can be used to access static variables, static methods, cast, and construct object. They are used wherever a Java type would be used in the Java syntax such as creating an array or accessing the class instance. These class wrappers are customized in Python to allow a direct mapping from Java concepts to Python one. These are all created dynamically corresponding to each Java class. For most of this document we will refer to these simply as a "class". (`java.lang.Object`_, `java.lang.String`_, etc) Many wrappers are customized to match Python abstract base classes ABC (`java.util.List`_, `java.util.Map`_) Object Instances These are Java objects. They operate just like Python objects with Java public fields mapped to Python attributes and Java methods to Python methods. For this document we will refer to an object instance simply as an "object". The object instance is split into two halves. The Python portion is referred to as the "handle" that points the Java "instance". The lifetime of the "instance" is tied to the handle thus Java objects do not disappear until the Python handle is disposed of. Objects can be cast_ to match the required type and hold methods_ and fields. `Primitive types`_ Each of the 8 Java primitive types are defined. These are used to cast to a Java type or to construct arrays. (`JBoolean`_, `JChar`_, `JByte`_, `JShort`_, `JInt`_, `JLong`_, `JFloat`_, and `JDouble`_) Decorators Java has a number of keywords such as extending a class or implementing an interface. Those pieces of meta data can't directly be expressed with the Python syntax, but instead have been been expressed as annotations that can be placed on classes or functions to augment them with Java specific information. (`@JImplements`_, `@JOverride`_, `@JImplementationFor`_) Mapping Java syntax to Python Many Java concepts like try with resources can be mapped into Python directly (as the ``with`` statement), or Java try, throw, catch mapping to Python try, raise, except. Others such as synchronize do not have an exact Python match. Those have instead been mapped to special functions that interact with Python syntax.. (synchronized_, `with`, `try`, import_) JVM control functions The JVM requires specific actions corresponding to JNI functions in order to start, shutdown, and define threading behavior. These top level control functions are held in the ``jpype`` module. (startJVM_, shutdownJVM_) We will detail each of these concepts in greater detail in the later sections. Name mangling ============= When providing Java package, classes, methods, and fields to Python, there are occasionally naming conflicts. For example, if one has a method called ``with`` then it would conflict with the Python keyword ``with``. Wherever this occurs, JPype renames the offending symbol with a trailing under bar. Java symbols with a leading or trailing under bars are consider to be privates and may not appear in the JPype wrapper entirely with the exception of package names. The following Python words will trigger name mangling of a Java name: =========== =========== ============= =========== ========== ``False`` ``None`` ``True`` ``and`` ``as`` ``async`` ``await`` ``def`` ``del`` ``elif`` ``except`` ``exec`` ``from`` ``global`` ``in`` ``is`` ``lambda`` ``nonlocal`` ``not`` ``or`` ``pass`` ``print`` ``raise`` ``with`` ``yield`` =========== =========== ============= =========== ========== JPype Types *********** Both Java and Python have a concept of a type. Every variable refers to an object which has a defined type. A type defines the data that the variable is currently holding and how that variable can be used. In this chapter we will learn how Java and Python types relate to one another, how to create import types from Java, and how to use types to create Java objects. Stay strong in a weak language ============================== Before we get into the details of the types that JPype provides, we first need to contrast some of the fundamental language differences between Java and Python. Python is inherently a weakly typed language. Any variable can take any type and the type of a particular variable can change over the lifetime of a program. Types themselves can be mutable as you can patch an existing type to add new behaviors. Python methods can in principle take any type of object as an argument, however if the interface is limited it will produce a TypeError to indicate a particular argument requires a specific type. Python objects and classes are open. Each class and object is basically a dictionary storing a set of key value pairs. Types implemented in native C are often more closed and thus can't have their method dictionaries or data members altered arbitrarily. But subject to a few restrictions based implementation, it is pretty much the wild west. In contrast, Java is a strongly typed language. Each variable can only take a value of the specified class or a class that derives from the specified class. Each Java method takes only a specific number and type of arguments. The type and number are all checked at compile type to ensure there is little possibility of error. As each method requires a specific number and type of arguments, a method can be overloaded by having two different implementations which take a different list of types sharing the same method name. A primitive variable can never hold an object and it can only be converted to or from other primitive types unless it is specifically cast to that type. Java objects and classes are completely closed. The methods and fields for a particular class and object are defined entirely at compile time. Though it is possible create classes with a dictionary allowing expansion, this is not the Java norm and no standard mechanism exists. Thus we need to introduce a few Java terms to the Python vocabulary. These are "conversion" and "cast". Java conversions ---------------- A conversion is a permitted change from an object of one type to another. Conversions have three different degrees. These are: exact, derived, implicit, and explicit. Exact conversions are those in which the type of an object is identical. In Java each class has only one definition thus there is no need for an exact conversion. But when dealing with Python we have objects that are effectively identical for which exact conversion rules apply. For example, a Java string and a Python string both bind equally well to a method which requires a string, thus this is an exact conversion for the purposes of bind types. The next level of conversion is derived. A derived class is one which is a descends from a required type. It is better that implicit but worse than exact. If all of the types in a method match are exact or derived then it will override a method in which one argument is implicit. The next level of conversion is implicit. An implicit conversion is one that Java would perform automatically. Java defines a number of other conversions such as converting a primitive to a boxed type or from a boxed type back to a primitive as implicit conversions. Python conversions defined by the user are also considered to be implicit. Of course not every cast is safe to perform. For example, converting an object whose type is currently viewed as a base type to a derived type is not performed automatically nor is converting from one boxed type to another. For those operations the conversion must be explicitly requested, hence these are explicit conversions. In Java, a cast is requested by placing the type name in parentheses in front of the object to be cast. Python does not directly support Java casting syntax. To request an explicit conversion an object must be "cast" using a cast operator @. Overloaded methods with an explicit argument will not be matched. After applying an explicit cast, the match quality can improve to exact or derived depending on the cast type. Not every conversion is possible between Java types. Types that cannot be converted are considerer to be conversion type "none". Details on the standard conversions provided by JPype are given in the section `Type Matching`_. .. _cast: Java casting ------------ To access a casting operation we use the casting ``JObject`` wrapper. For example, ``JObject(object, Type)`` would produce a copy with specificed type. The first argument is the object to convert and the second is the type to cast to. The second argument should always be a Java type specified using a class wrapper, a Java class instance, or a string. Casting will also add a hidden class argument to the resulting object such that it is treated as the cast type for the duration of that variable lifespan. Therefore, a variable create by casting is stuck as that type and cannot revert back to its original for the purposes of method resolution. The object construction and casting are sometimes a bit blurry. For example, when one casts a sequence to a Java list, we will end up constructing a new Java list that contains the elements of the original Python sequence. In general JPype constructors only provide access the Java constructor methods that are defined in the Java documentation. Casting on the other hand is entirely the domain of whatever JPype has defined including user defined casts. As ``JObject`` syntax is long and does not look much like Java syntax, the Python matmul operator is overloaded on JPype types such that one can use the ``@`` operator to cast to a specific Java type. In Java, one would write ``(Type)object`` to cast the variable ``object`` to ``Type``. In Python, this would be written as ``Type@object``. This can also be applied to array types ``JLong[:]@[1,2,3]``, collection types ``Iterable@[1,2,3]`` or Java functors ``DoubleUnaryOperator@(lambda x:x*2)``. The result of the casting operator will be a Java object with the desired type or raise a ``TypeError`` if the cast or conversion is not possible. For Python objects, the Java object will generally be a copy as it is not possible to reflect changes in an array back to Python. If one needs to retrieve the resulting changes keep a copy of the converted array before passing it. For an existing Java object, casting changes the resolution type for the object. This can be very useful when trying to call a specific method overload. For example, if we have a Java ``a=String("hello")`` and there were an overload of the method ``foo`` between ``String`` and ``Object`` we would need to select the overload with ``foo(java.lang.Object@a)``. .. _JObject: Casting is performed through the Python class ``JObject``. JObject is called with two arguments which are the object to be cast and the type to cast too. The cast first consults the conversion table to decide if the cast it permitted and produces a ``TypeError`` if the conversion is not possible. ``JObject`` also serves as a abstract base class for testing if an object instance belongs to Java. All objects that belong to Java will return true when tested with ``isinstance``. Like Python's sequence, JObject is an abstract base class. No classes actual derive from ``JObject``. .. _null: Of particular interest is the concept of Java ``null``. In Java, null is a typeless entity which can be placed wherever an object is taken to indicate that the object is not available. The equivalent concept in Python is ``None``. Thus all methods that accept any object type that permit a null will accept None as an augment with implicit conversion. However, sometime it is necessary to pass an explicit type to the method resolution. To achieve this in JPype use ``Type@None`` which will create a null pointer with the desired type. To test if something is null we have to compare the handle to None. This unfortunately trips up some code quality checkers. The idiom in Python is ``obj is None``, but as this only matches things that Python considers identical, we must instead use ``obj==None``. Casting ``None`` is use to specify types when calling between overloads with variadic arguments such as ``foo(Object a)`` and ``foo(Object... many)``. If we want to call ``foo(None)`` is is ambiguous whether we intend to call the first with a null object or the second with a null array. We can resolve the ambiguity with ``foo(java.lang.Object@None)`` or ``foo(java.lang.Object[:]@None)`` Type enforcement appears in three different places within JPype. These are whenever a Java method is called, whenever a Java field is set, and whenever Python returns a value back to Java. .. _methods: Method resolution ================= Because Java supports method overloading and Python does not, JPype wraps Java methods as a "method dispatch". The dispatch is a collection of all of the methods from class and all of its parents which share the same name. The job of the dispatch is chose the method to call. Enforcement of the strong typing of Java must be performed at runtime within Python. Each time a method is invoked, JPype must match against the list of all possible methods that the class implements and choose the best possible overload. For this reason the methods that appear in a JPype class will not be the actual Java methods, but rather a "dispatch" whose job is deciding which method should be called based on the type of the provided arguments. If no method is found that matches the provided arguments, the method dispatch will produce a ``TypeError``. This is the exact same outcome that Python uses when enforcing type safety within a function. If a type doesn't match a ``TypeError`` will be produced. Dispatch example ---------------- When JPype is unable to decide which overload of a method to call, the user must resolve the ambiguity. This is where casting comes in. Take for example the ``java.io.PrintStream`` class. This class has a variant of the print and println methods! So for the following code: .. code-block:: python java.lang.System.out.println(1) JPype will automatically choose the ``println(long)`` method, because the Python int matches exactly with the Java long, while all the other numerical types are only "implicit" matches. However, if that is not the version you wanted to call you must cast it. In this case we will use a primitive type to construct the correct type. Changing the line thus: .. code-block:: python java.lang.System.out.println(JByte(1)) # <--- wrap the 1 in a JByte This tells JPype to choose the byte version. When dealing with Java types, JPype follows the standard Java matching rules. Types can implicitly grow to larger types but will not shrink without an explicit cast. Primitive Types =============== Unlike Python, Java makes a distinction between objects and primitive data types. Primitives represent the minimum data that can be manipulated by a computer. These stand in contrast to objects which have the ability to contain any combination of data types and object within themselves, and can be inherited from. Java primitives come in three flavors. The logical primitive ``boolean`` can only take the logical value true and false. The textual primitive ``char`` represents one character in a string. Numerical primitives are intended for fixed point or floating point calculations. Numerical primitives come in many sizes depending on how much storage is required. In Java, integer numerical primitives are always signed and thus can only reach half their range in terms of bits up or down relative to their storage size. JPype has mapped each of the primitive types into Python classes. To avoid conflicts with Python, JPype has named each primitive with a capital letter ``J`` followed by the primitive name starting with an upper case letter. .. _JBoolean: JBoolean A boolean is the logical primitive as it can only take values ``True`` and ``False``. It should properly be an extension of the Python concept ``bool`` but that type is not extendable. Thus instead it must inherit from ``int``. This type is rarely seen in JPype as the values ``True`` and ``False`` are considered an exact match to ``JBoolean`` argument. Methods which return a ``JBoolean`` will always return a Python ``bool`` rather than a Java primitive type. .. _JChar: JChar A character is the textual primitive that corresponds to exactly one character in a string. Or at least that was the concept at the time. Java characters can only represent 16 bits. But there are currently 143,924 defined characters in Unicode. Thus, there are certain characters that can only be represented as two Unicode characters. The textual primitives are not intended to perform numerical functions, but are instead encoded. As per the old joke, what does `1` plus `1` equal? Which of course the correct answer is `b`. As such characters should not be treated as just another unsigned short. Python has no concept of a textual only type. Thus when returning a character type, we instead return a string length 1. ``JChar`` supports the Java numerical operations, but just as in Java it will automatically promote to a Python ``int`` when used in a numerical operation. There are of course lots of useful mathematical operations that can be performed on textual primitives, but doing so risks breaking the encoding and can result in uninterpretable data. .. _JByte: .. _JShort: .. _JInt: .. _JLong: JByte, Short, Int, Long These types represent fixed point quantities with ranges of 8, 16, 32, and 64 bits. Each of these type inherit from a Python ``int`` type. A method or field returning an integer primitive will return a type derived from ``int``. Methods accepting an integer primitive will take either an Java integer primitive or a Python ``int`` or anything that quacks like a ``int`` so long as it can be converted into that primitive range without truncation. .. _JFloat: .. _JDouble: JFloat, JDouble These two types hold floating point and correspond to either single point (32 bit) or double point (64 bit) precision. Python does not have a concept of precision and thus both of these derive from the Python type ``float``. As per Java rules numbers greater than the range correspond to the values of positive and negative infinity. Conversions from Python types are ranged check and will produce a ``OverflowError`` if the value doesn't fit into the request types. If an overflow error is not desired, first cast the value into the request size prior to calling. Methods that return a Java floating point primitive will always return a value derived from ``float``. The classes for Java primitives are closed and should not be extended. As with all Java values any information attached to the Python representation is lost when passing that value to Java. Objects & Classes ================= In contrast to primitive data type, objects can hold any combination of primitives or objects. Thus they represent structured data. Objects can also hold methods which operate on that data. Objects can inherit from one another. However unlike Python, Java objects must have a fixed structure which defines its type. These are referred to the object's class. Here is a point of confusion. Java has two different class concepts: the class definition and the class instance. When you import a class or refer to a method using the class name you are accessing the class definition. When you call ``getClass`` on an object it returns a class instance. The class instance is a object whose structure can be used to access the data and methods that define the class through reflection. The class instance cannot directly access the fields or method within a class but instead provides its own interface for querying the class. For the purposes of this document a "class" will refer to the class definition which corresponds to the Python concept of a class. Wherever the Java reflection class is being referred to we will use the term "class instance". The term "type" is synonymous with a "class" in Java, though often the term "type" is only used when inclusively discussing the type of primitives and objects, while the term "class" generally refers to just the types associated with objects. All objects in Java inherit from the same base class ``java.lang.Object``, but Java does not support multiple inheritance. Thus each class can only inherit from a single parent. Multiple inheritance, mix-ins, and diamond pattern are not possible in Java. Instead Java uses the concept of an interface. Any Java class can inherit as many interfaces as it wants, but these interfaces may not contain any data elements. As they do not contain data elements there can be no ambiguity as to what data a particular lookup. .. _JInterface: The meta class ``JInterface`` is used to check if a class type is an interface using ``isinstance``. Classes that are pure interfaces cannot be instantiated, thus, there is not such thing as an abstract instance. Therefore, every Java object should have Objects cannot actual be pure interfaces. To represent this in Python every interface inherits ``java.lang.Object`` methods even through it does not have ``java.lang.Object`` as a parent. This ensures that anonymous classes and lambdas have full object behavior. Classes ------- In JPype, Java classes are instances of the Python ``type`` and function like any ordinary Python class. However unlike Python types, Java classes are closed and cannot be extended. To enforce extension restrictions, all Java classes are created from a special private meta class called ``_jpype._JClass``. This gatekeeper ensures that the attributes of classes cannot be changed accidentally nor extended. The type tree of Java is fixed and closed. All Java classes have the following functionality. Class constructor The class constructor is accessed by using the Python call syntax ``()``. This special method invokes a dispatch whenever the class is called as a function. If an matching constructor is found a new Java instance is created and a Python handle to that instance is returned. In the case of primitive types, the constructor creates a Java value with the exact type requested. Get attribute The Python ``.`` operator gets an attribute from a class with a specified name. If no method or field exists a ``AttributeError`` will be raised. For public static methods, the getattr will produce a Python descriptor which can be called to invoke the static method. For public static fields, a Python descriptor will be produced that allows the field to be get or set depending on whether the field is final or not. Public instance methods and instance fields will produce a function that can be applied to a Java object to execute that method or access the field. Function accessors are non-virtual and thus they can provide access to behaviors that have been hidden by a derived class. Set attribute In general, JPype only allows the setting of public non-final fields. If you attempt to set any attribute on an object that does not correspond to a settable field it will produce an ``AttributeError``. There is one exception to this rule. Sometime it is necessary to attach addition private meta data to classes and objects. Attributes that begin with an underbar are consider to be Python private attributes. Private attributes handled by the default Python attribute handler allowing these attributes to be attached to to attach data to the Python handle. This data is invisible to Java and it is retained only on the Python instance. If an object with Python meta data is passed to Java and Java returns the object, the new Python handle will not contain any of the attached data as this data was lost when the object was passed to Java. ``class_`` Attribute For Java classes there is a special attribute called ``class``. This is a keyword in Python so `name mangling`_ applies. This is a class instance of type ``java.lang.Class``. It can be used to access fields and methods. Inner classes For methods and fields, public inner classes appear as attributes of the class. These are regular types that can be used to construct objects, create array, or cast. String The Java method ``toString`` is mapped into the Python function ``str(obj)``. Equality The Java method ``equals()`` has been mapped to Python ``==`` with special augmentations for null pointers. Java ``==`` is not exposed directly as it would lead to numerous errors. In principle, Java ``==`` should map to the Python concept of ``is`` but it is not currently possible to overload Python in such a way to achieve the desired effect. Hash The Java method ``hashCode`` is mapped to Python ``hash(obj)`` function. There are special augmentations for strings and nulls. Strings will return the same hash code as returned by Python so that Java strings and Python strings produce the same dictionary lookups. Null pointers produce the same hash value as None. Java defines ``hashCode`` on many objects including mutable ones. Often the ``hashCode`` for a mutable object changes when the object is changed. Only use immutable Java object (String, Instant, Boxed types) as dictionary keys or risk undefined behavior. Java objects are instances of Java classes and have all of the methods defined in the Java class including static members. However, the get attribute method converts public instance members and fields into descriptors which act on the object. Now that we have defined the basics of Java objects and classes, we will define a few special classes that operate a bit differently. Array Classes ------------- In Java all arrays are also objects, but they cannot define any methods beyond a limited set of Java array operations. These operations have been mapped into Python to their closest Python equivalent. Arrays also have a special type factory to produce them. In principle one can create an array class using ``JClass`` but the signature required would need to use the proper name as required for the Java method ``java.lang.Class.forName``. Instead we call the factory to create a new type to use. .. _JArray: The signature for JArray is ``JArray(type, [dims=1])``. The type argument accepts any Java type including primitives and constructs a new array class. This class can be used to create new instances, cast, or as the input to the array factory. The resulting object has a constructor method which take either a number, which is the desired size of the array, or a sequence which hold the elements of the array. If the members of the initializer sequence are not Java members then each will be converted. If any element cannot be converted a ``TypeError`` will be raised. As a shortcut the ``[]`` operator can be used to specify an array type or an array instance. For example, ``JInt[5]`` will allocate an array instance of Java ints with length 5. ``JInt[:]`` will create a type instance with an unspecific length which can be used for the casting operator. To create an array instance with multiple dimensions we would use ``JInt[5,10]`` which would create a rectangular array which was 5 by 10. To create a jagged array we would substitute ``:`` for the final dimensions. So ``JInt[5,:]`` is a length 5 array of an array of ``int[]``. Multidimensional array types are specificed like ``JInt[:,:,:]`` would be a Java type ``int[][][]``. This applied to both primitive and object types. JArray is an abstract base class for all Java classes that are produced. Thus, one can test if something is an array class using ``issubclass`` and if Java object is an array using ``isinstance``. Java arrays provide a few additional Python methods: Get Item Arrays are of course a collection of elements. As such array elements can be accessed using the Python ``[]`` operator. For multidimensional arrays JPype uses Java style access with a series of index operations such as ``jarray[4][2]`` rather than NumPy like multidimensional access. Get Slice Arrays can be accessed using a slice like a Python list. The slice operator is ``[start:stop:step]``. It should be noted that array slice are in fact views to the original array so any alteration to the slice will affect the original array. Array slices are cloned when passed back to Java. To force a clone immediately, use the ``clone`` method. Please note that applying the slice operator to a slice produces a new slice. Thus there can sometimes be an ambiguity between multidimensional access and repeated slicing. Set Item Array items can be set using the Python ``[]=`` operator. Set Slice Multiple array items can be set using a slice assigned with a sequence. The sequence must have the same length as the slice. If this condition is not met, an exception will be raised. If the items to be transferred are a buffer, then a faster buffer transfer assignment will be used. When buffer transfers are used individual elements are not checked for range, but instead cast just like NumPy. Thus, if we have the elements we wish to assign to the array contained within a NumPy array named ``na`` we can transfer all of them using ``jarray[:] = na``. Buffer transfer Buffer transfers from a Java array also work for primitive types. Thus we can simply call the Python ``memoryview(jarray)`` function to create a buffer that can be used to transfer any portion of a Java array out. Memory views of Java arrays are not writable. For each Java arrays can be used as the input to a Python for statement. To iterate each element use ``for elem in jarray:``. They can also be used in list comprehensions. Clone Java arrays can be duplicated using the method clone. To create a copy call ``jarray.clone()``. This operates both on arrays and slice views. Length Arrays in Java have a defined an immutable length. As such the Python ``len(array)`` function will produce the array length. However, as that does not match Java expectations, JPype also adds an attribute for length so that Java idiom ``jarray.length`` also works as expected. In addition, the Java class ``JChar[]`` has some addition customizations to help work better with string types. Java arrays are currently missing some of the requirements to act as a ``collections.abc.Sequence``. When working with Java arrays it is also useful to use the Java array utilities class ``java.util.Arrays`` as it has many methods that provide additional functionality. Java arrays do not support any additional mathematical operations at this time. Creating a Java array is also required when pass by reference syntax is required. For example, if a Java function takes an array, modifies it and we want to retrieve those values. In Java, all parameters are pass by value, but the contents of a container like an array can be modified which gives the appearance of pass by reference. For example. .. code-block:: java public void modifies(int[] v) { for (int i=0; i