Adding features

General principles

niimpy is an open source project and general open source contribution guidelines apply - there is no need for us to repeat them right now. Please use Github for communication.

Contributions are welcome and encouraged. * You don’t need to be perfect. Suggest what you can and we will help it improve.

Adding an analysis

  • Please add documentatation to a sensor page when you add a new analysis. This should include enough description so that someone else can understand and reproduce all relevant features - enough to describe the method for a scientific article.

  • Please add unit tests which test each relevant feature (and each claimed method feature) with a minimal example. Each function can have multiple tests. For examples of unit tests, see below or niimpy/test_screen.py. You can create some sample data within each test module which can be used both during development and for tests.

Common things to note

  • You should always use the DataFrame index to retrieve data/time values, not the datetime column (which is a convenience thing but not guaranteed to be there).

  • Don’t require datetime in your input

  • Have any times returned in the index (unless each row needs multiple times, then do what you need)

  • Don’t fail if there are extra columns passed (or missing some non-essential columns). Look at what columns/data is passed and and use that, but don’t do anything unexpected if someone makes a mistake with input data

  • Group by ‘user’ and ‘device’ columns if they are present in the input

  • Use niimpy.util._read_sqlite_auto function for getting data from input

  • Use niimpy.filter.filter_dataframe to do basic initial filterings based on standard arguments.

  • The Zen of Python is always good advice

Improving old functions

  • Add tests for existing functionality

  • For every functionality it claims, there should be a minimal test for it.

  • Use read._get_dataframe and filter.filter_dataframe to handle standard arguments

  • Don’t fail if unnecessary columns are not there (don’t drop unneeded columns, select only the needed ones).

  • Make sure it uses the index, not the datetime column. Some older functions mays still expect it so we have a difficult challenge.

  • Improve the docstring of the function: we use the numpydoc format

  • Add a documentation page for these sensors, document each function and include an example.

  • Document what parameters it groups by when analyzing

  • For example an ideal case is that any ‘user’ and ‘device’ columns are grouped by in the final output.

  • When there are things that don’t work yet, you can put a TODO in the docstring to indicate that someone should come back to it later.

Example unit test

You can read about testing in general in the CodeRefinery testing lesson.

First you would define some sample data. You could reuse existing data (or data from niimpy.sampledata), but if data is reused too much then it becomes hard to improve test B because it will affect the data of test A. (do share data when possible but split it when it’s relevant).

@pytest.fixture
def screen1():
    return niimpy.read_csv(io.StringIO("""\
time,screen_status
0,1
60,0
"""))

Then you can make a test function:

def test_screen_off(screen1):
    off = niimpy.preprocess.screen_off(screen1)
    assert pd.Timestamp(60,  unit='s', tz=TZ) in off.index

assert statemnts run the tested functions - when there are errors pytest will provide much more useful error messages than you might expect. You can have multiple asserts within a function, to test multiple things.

You run tests with pytest niimpy/ or pytest niimpy/test_screen.py. You can limit to certain tests with -k and engage a debugger on errors with --pdb.

Documentation notes

  • You can use Jupyter or ReST. ReST is better for narritive documentation.

[ ]: