{
  "id": "jonnor/micropython-npyfile",
  "name": "micropython-npyfile",
  "prettyName": "micropython-npyfile",
  "description": "Numpy .npy file support for MicroPython - read/write/streaming",
  "version": "",
  "author": "jonnor",
  "license": "MIT",
  "tags": [],
  "status": "VALID",
  "official": false,
  "source": "https://github.com/jonnor/micropython-npyfile",
  "updatedAt": "2025-06-28T07:00:19.768Z",
  "page": "https://checkmim.com/packages/jonnor+micropython-npyfile",
  "createdAt": "2025-05-27T04:30:06.767Z",
  "submittedBy": "oliverr",
  "install": {
    "mpremote": "mpremote mip install github:jonnor/micropython-npyfile",
    "mip": "import mip; mip.install(\"github:jonnor/micropython-npyfile\")"
  },
  "readme": "\n[![Tests](https://github.com/jonnor/micropython-npyfile/actions/workflows/tests.yaml/badge.svg?branch=master)](https://github.com/jonnor/micropython-npyfile/actions/workflows/tests.yaml)\n\n# micropython-npyfile\n\nSupport for [Numpy files (.npy)](https://numpy.org/doc/stable/reference/generated/numpy.lib.format.html) for [MicroPython](https://micropython.org/).\nSimple persistence of multi-dimensional numeric array data, and interoperability with Numpy/CPython et.c.\n\nThis is useful for example for sensor data, time-series, audio, images, bitmaps, et.c,\nwith applications in IoT systems, robotics, etc.\n\nWas initially written to be used with [emlearn-micropython](https://github.com/emlearn/emlearn-micropython),\na Machine Learning and Digital Signal Processing library for MicroPython.\n\nIf your data is homogenous numeric data, then .npy files is as superior alternativet to comma-separated values (CSV/TSV etc),\nboth in terms of storage space and processing time.\n\n#### Features\n\n- Reading & writing .npy files with numeric data (see below for Limitations)\n- Streaming/chunked reading & writing\n- No external dependencies. Uses [array.array](https://docs.micropython.org/en/latest/library/array.html)\n- Written in pure Python. Compatible with CPython, CircuitPython, et.c.\n\n\n## Installing\n\nThis package can be installed using [mip](https://docs.micropython.org/en/latest/reference/packages.html#installing-packages-with-mip).\n\nFor example:\n\n```bash\nmpremote mip install github:jonnor/micropython-npyfile\n```\n\nOr just copy the `npyfile.py` file to your MicroPython device.\n\n## Usage\n\n#### Save a file (simple)\n\n```python\n\nimport array\nimport npyfile\n\nshape = (10, 4)\ndata = array.array('f', (1.0 for _ in range(shape[0]*shape[1])))\n\nnpyfile.save('mydata.npy', data, shape)\n```\n\n#### Load a file (simple)\n\n```python\n\nimport npyfile\nshape, data = npyfile.load('mydata.npy')\n\nprint(shape)\nprint(data)\n```\n\n#### Streaming read\n\nStreaming/chunked reading can be used to keep memory usage low.\n\n```python\nimport npyfile\n\nwith npyfile.Reader('mydata.npy') as reader:\n\n    # Metadata available on the reader object\n    print(reader.shape, reader.typecode, reader.itemsize)\n\n    # NOTE: assumes input is 2d. Pick chunksize in another way if not\n    chunksize = reader.shape[1]\n    for chunk in reader.read_data_chunks(chunksize):\n        print(len(chunk), chunk)\n```\n\nMore examples:\n\n- Streaming matching data from two files: [two_streams.py](./examples/digits/two_streams.py)\n\n#### Streaming write\n\nStreaming/chunked writing can be used to keep memory usage low.\n\n```python\nimport npyfile\nimport array\n\nwith npyfile.Writer('output.npy', shape=(5, 3), typecode='f') as writer:\n    chunk = array.array('f', (1.1, 2.2, 3.3))\n\n    for i in range(5):\n        writer.write_values(chunk, typecode='f')\n\ndata, shape = npyfile.load('output.npy')\nprint(shape)\nprint(data)\n```\n\nMore examples:\n\n- Streaming read and write: [streaming_readwrite.py](./examples/digits/streaming_readwrite.py)\n- Implementation of `npyfile.save()`, in [npyfile.py](./npyfile.py)\n \n\n#### Reading .npz files\n\n.npz files are ZIP archives.\nThey can be read using an implementation of the zipfile module,\nsuch as [jonnor/micropython-zipfile](https://github.com/jonnor/micropython-zipfile).\nBoth uncompressed and DEFLATE compressed files are supported.\n\nFor example code, see [read_npz.py](./examples/read_npz.py).\n\n#### Saving portable files with numpy\n\nnumpy.save defaults to allowing pickle support.\nThis makes is very easy to accidentially save numpy.array\nobjects that use pickle for serialization, instead of just numeric data.\nSuch files will not be loadable by `npyfile` (or most other .npy readers).\n\nTo avoid this, when saving numeric arrays, always use:\n\n```python\nnumpy.save(PATH, DATA, allow_pickle=False)\n```\n\n\n## Limitations\n\n- Only little-endian is supported, not big-endian\n- Only C data order is supported, not Fortran\n- Strings are not supported\n- Complex numbers not supported\n- Pickled data is not supported\n\nNOT TESTED on unsupported/malformed/malicious inputs.\n\n## TODO \nContributions welcomed!\n\nTODO:\n\n- Add tests for unsupported/malformed inputs\n\n## Developing\n\n#### Running tests on host\n\nInstall the Unix/Window port of MicroPython. Then run:\n\n```\nMICROPYPATH=./ micropython tests/test_npyfile.py\n```\n\nThe tests can also be ran under CPython\n```\nPYTHONPATH=./ python tests/test_npyfile.py\n```\n\n#### Running tests on device\n\nConnect a MicroPython device via USB.\n\nCopy over the data\n```\nmpremote cp npyfile.py :\nmpremote -r cp tests/ :\nmpremote run tests/test_npyfile.py\n```\n\n"
}