Adding features

General principles

Niimpy is an open source project and general open source contribution guidelines apply - there is no need for us to repeat them right now. Please use Github for communication.

Contributions are welcome and encouraged.

  • You don’t need to be perfect. Suggest what you can and we will help it improve.

Adding functionality

  • Please add documentation to each new function using docstrings. This should include enough description so that someone else can understand and reproduce all relevant features - enough to describe the method for a scientific article.

  • Please add unit tests which test each relevant feature (and each claimed method feature) with a minimal example. Each function can have multiple tests. For examples of unit tests, see below or niimpy/test_screen.py. You can create some sample data within each test module which can be used both during development and for tests.

Common things to note

  • You should always use the DataFrame index to retrieve data/time values, not the datetime column (which is a convenience thing but not guaranteed to be there).

  • Don’t require datetime in your input

  • Have any times returned in the index (unless each row needs multiple times, then do what you need)

  • Don’t fail if there are extra columns passed (or missing some non-essential columns). Look at what columns/data is passed and and use that, but don’t do anything unexpected if someone makes a mistake with input data

  • Group by ‘user’ and ‘device’ columns if they are present in the input

  • Resample by the time index, using given resample arguments (or a default value).

  • Use niimpy.util._read_sqlite_auto function for getting data from input

  • Use niimpy.filter.filter_dataframe to do basic initial filterings based on standard arguments.

  • The Zen of Python is always good advice

Improving old functions

  • If you find a bug, problem or potential enhancement, let us know in an Issue on the Niimpy GitHub page.

  • Before sinking time into fixing the issue or improving Niimpy, discuss with us on the Issue. This ensures that no-one else is working on it and that we can help you with the process.

  • To suggest a change, preferably fork the repository and create a pull request.

  • Add tests for existing functionality

  • For every functionality Niimpy claims, there should be a minimal test for it.

  • Use read._get_dataframe and filter.filter_dataframe to handle standard arguments

  • Don’t fail if unnecessary columns are not there (don’t drop unneeded columns, select only the needed ones).

  • Make sure it uses the index, not the datetime column.

  • Improve the docstring of the function: we use the numpydoc format

  • Add a documentation page for each sensor, document each function and include an example.

  • Document what parameters the function groups by when analyzing

  • For example an ideal case is that any ‘user’ and ‘device’ columns are grouped by in the final output.

Example unit test

You can read about testing in general in the CodeRefinery testing lesson.

First you would define some sample data. You could reuse existing data (or data from niimpy.sampledata), but if data is reused too much then it becomes hard to improve test B because it will affect the data of test A. (do share data when possible but split it when it’s relevant).

@pytest.fixture
def screen1():
    return niimpy.read_csv(io.StringIO("""\
time,screen_status
0,1
60,0
"""))

Then you can make a test function:

def test_screen_off(screen1):
    off = niimpy.preprocess.screen_off(screen1)
    assert pd.Timestamp(60,  unit='s', tz=TZ) in off.index

assert statemnts run the tested functions - when there are errors pytest will provide much more useful error messages than you might expect. You can have multiple asserts within a function, to test multiple things.

You run tests with pytest niimpy/ or pytest niimpy/test_screen.py. You can limit to certain tests with -k and engage a debugger on errors with --pdb.

Documentation notes

  • You can use Jupyter or ReST. ReST is better for narritive documentation.