The Apache SystemML 1.1.0 release was approved on March 28, 2018. The release includes enhancements, features, and additions as listed below.
New Builtin Functions/Operations
ifelse
assert
- Deep learning builtin functions:
avg_pool
, avg_pool_backward
- Second-order
eval
- Accumulator operator +=
- Bitwise operators not, and, or, xor, & LShift, Rshift
- Logical operator support over matrices AND/OR/NOT/XOR
Additional Layers in the NN library
- Average pooling
- Upsampling
- Low-rank fully connected
New Capabilities/Features
- Dense matrix blocks >16GB, and operations
- Enable additional ParFor result aggregation operations
- UDFs callable in expressions
- Zero rows/columns matrices and updated operations such as removeEmpty
- Matrix-matrix multiplication over compressed matrices
- Extended Caffe2DML and Keras2DML APIs
Compiler & Runtime
- Use common thread pool
- Single-precision support for native conv2d and mm operations
- Improved nnz maintenance, runtime propogation and memory management
- Robustness for matrices with larger than int dimensions
- [Experimental] Codegen extensions: operation support, extended optimizer, see SYSTEMML-2065
- sparse left indexing, sparse reshape, ultra-sparse transpose, ultra-sparse rand, binary in-place operations, sparse relu backward, maxpooling, sparse im2col, ultra-sparse conv2d, read of short-wide sparse matrices, avoid unnecessary evictions, lock-free statistics maintenance, spark cpmm, spark aggregates, spark reshape, spark binary ops, etc.
Bug Fixes
- in APIs, performance, optimizer, runtime, GPU backend, Spark backend
Deprecate
- Support for Spark 2.1 / 2.2 (make switch to newer ANTLR version)