pmesh.coarray module¶

CoArray/MPI in Python

This is a failed attempt to implement the CoArray 1.0 standard:

http://caf.rice.edu/documentation/index.html

The implementation uses the MPI 1.0 subset, mostly because I am not quite familar with MPI.Win objects and how they interact with numpy. mpi4py also has a few test blacklists for versions of openmpi that looked a bit scary.

The main entry object is coarray a subclass of numpy.ndarray,

A coarray can be created via coarray.zeros or coarray.fromarray

A sync operation must be invoked to issue data transport. The standard does not expect any data transfer before sync is called.

Comparing to CoFortran, the use of [] and () is inverted, because in Python [] is for slicing.

Fortran methods (thisimage, sync) are implemented as properties.

The standard mentioned rank, which is ndims. co-rank becomes condims.

default attributes from numpy refers to local properties.

No methods are provided for the global shape, global ndims.

What doesn’t work:

Currently the co-ndims is limited to 1:

This can be done either use MPI’s Cart Comm, or do some simple encoding. Because MPI.Comm doesn’t quite work with the life cycle of python objects, requiring explicit MPI.Comm.Free, reshaping the co-ndims will be difficult if we hold MPI Cart Comm objects inside coarrays; unless we don’t care about correctness.

Broadcast and Gather are not supported.

Treating the coarray dimension as an array dimension (co-transpose) doesn’t work neither.

Currently there is no way to notify the source rank which coarray to use for the send operation.

We need to add all coarrays to a central repo to do that; and then may need to crawl a dependency graph. I think there will be a lot of trouble ahead.

But things may be simplified if we make use of the fact that all operations are ran symmetrically.

Yu Feng <rainwoodman@gmail.com>