AHF_Datalogger

The datalogger implemented in the code is determined through the jsn file for the relevant task. Currently, AHF_DataLogger_textMySql is used for all tasks; however, SQL logging is currently disabled due to compatibility issues.

Note: hdf5 logs are created independent of this module, in AHF_Stimulus_Laser.py

AHF_Datalogger.AHF_DataLogger module

class AHF_DataLogger.AHF_DataLogger(taskP, settingsDictP)

Bases: AHF_Base.AHF_Base

A Data Logger provides an interface to save task data, and to save and load mouse configuration data. This can be either to text files, or to a database, or hd5 files, or some combination thereof. The data logger should also print status updates to the shell, but these don’t need to contain as much information. The brain imaging data is saved separately, but references to movie files should be saved by data logger. Similarly, other binary data(lever positons, anyone?) can be saved separately, by the Stimulator class, but binary file/posiiton can be saved as an event.

BUFFER_SIZE = 25

TO_FILE = 2

TO_SHELL = 1

clearTrackedValues(tag, eventKind, dictKey)

abstract configGenerator()

generates configuration data for each subject as(IDtag, dictionary) tuples from some kind of permanent storage such as a JSON file, or a database. Will be called when program is started, or restarted and settings need to be reloaded.

abstract getConfigData(tag)

returns a dictionary of data that was saved for this reference tag, in some permanent storage such as a JSON file Will be called when program is started, or restarted and settings need to be reloaded

abstract getMice()

returns a list of mice that are in the dictionary/database

getTrackedEvent(tag, eventKind, dictKey)

Returns the current value for the specified mouse, event, and key.

isChild = False

abstract makeLogFile()

Makes or opens a text log file, or a datbase, or whatever else needs doing. Called once before entering main loop of program. DataLogger may make a new file every day in NewDay function, if desired

abstract newDay()

At the start of a new day, it was customary for the text-based data logging to start new text files, and to make a precis of the day’s results into a a separate text file for easy human reading. This “quickStats” file should contain info for each mouse with rewards, head fixes, or tasks, and other Stimulator specific data, which Stimulator object will provide for each mouse just call the Stimulator class functions for each mouse to get a dictionary of results

abstract readFromLogFile(index)

Reads the log statement index lines prior to the current line. Returns the event and associated dictionary in a tuple.

resumeTracking(eventKind, dictKey)

Resumes previously started tracking.

abstract retireMouse(tag, reason)

store information about a mouse retirement in a referenced file

abstract saveNewMouse(tag, note, dictionary)

store a new mouse entry in a referenced file

startTracking(eventKind, dictKey, trackingType, size=0)

Begins tracking of the specified key for the specified event. Tracks as a circular buffer or daily totals.

stopTracking(eventKind, dictKey)

Halts previously started tracking.

abstract storeConfig(tag, dictionary, source='')

Stores configuration data, given as an IDtag, and dictionary for that tag, in some more permanent storage as a JSON text file, or a database or hd5 file, so it can be later retrieved by IDtag

trackingDict = {}

abstract writeToLogFile(tag, eventKind, eventDict, timeStamp, toShellOrFile=3)

The original standard text file method was 4 tab-separated columns, mouse tag, or 0 if no single tag was applicaple, unix time stamp, ISO formatted time, and event. Event could be anything. Now, every event has a kind, and every kind of event defines a dictionary. Main program calls writeToLogFile, as well as the Stimulator object For text based methods, event should be a dictionary for more complicated stimulator results, so an event can be more easily parsed during data analysis.

AHF_Datalogger.AHF_DataLogger_localsql module

class AHF_DataLogger_localsql.AHF_DataLogger_localsql(taskP, settingsDictP)

Bases: AHF_DataLogger.AHF_DataLogger

Data logger that uses the local mysql database. Used in case of unstable remote connections.

Mouse data is stored in a specified folder as text files, one text file per mouse containing JSON formatted configuration and performance data. These files will opened and updated after each exit from the experimental tube, in case the program needs to be restarted The file name for each mouse contains RFID tag 0-padded to 13 spaces: AHF_mouse_1234567890123.jsn

REQUESTED VALUES: “exit” for exits as event “lever_pull” as event for lever data “positions” as key for the lever positions in the event_dictionary of “lever_pull”

PSEUDO_MUTEX = 0

The class field PSEUDO_MUTEX helps prevent print statements from different places in the code(main vs callbacks) from executing at the same time, which leads to garbled output. Unlike a real mutex, execution of the thread is not halted while waiting on the PSEUDO_MUTEX, hence the loops with calls to sleep to allow the other threads of execution to continue while waiting for the mutex to be free. Also, read/write to the PSEUDO_MUTEX is not atomic; one thread may read PSEUDO_MUTEX as 0, and set it to 1, but in the interval between reading and writing to PSEUDO_MUTEX,another thread may have read PSEUDO_MUTEX as 0 and both threads think they have the mutex

static about()

Returns a brief message describing your sub-class, used when asking user to pick a sub-class of this class

configGenerator(settings)

Each configuration file has config data for a single subject. This function loads data from all of them in turn, and returning each as a a tuple of(tagID, dictionary)

static config_user_get(starterDict={})

static method that querries user for settings, with default responses from starterDict, and returns starterDict with settings as edited by the user.

defaultCage = 'cage1'

getConfigData(tag, source)

returns a dictionary of data that was saved for this reference tag, in some permanent storage such as a JSON file Will be called when program is started, or restarted and settings need to be reloaded

getFromDatabase(query, values)

getMice()

returns a list of mice that are in the dictionary/database

hardwareTest()

Tests functionality, gives user a chance to change settings. :Returns: bool – True if any settings have changed

localDatabase = 'raw_data'

localHost = 'localhost'

localPassword = '*********************'

localUser = 'pi'

makeLogFile()

Initiating database creation

newDay()

At the start of a new day, it was customary for the text-based data logging to start new text files, and to make a precis of the day’s results into a a separate text file for easy human reading. This “quickStats” file should contain info for each mouse with rewards, head fixes, or tasks, and other Stimulator specific data, which Stimulator object will provide for each mouse just call the Stimulator class functions for each mouse to get a dictionary of results

pingServers()

readFromLogFile(index)

Reads the log statement index lines prior to the current line. Returns the event and associated dictionary in a tuple.

retireMouse(tag, reason)

store information about a mouse retirement in a referenced file

saveNewMouse(tag, note, dictionary={})

store a new mouse entry in a referenced file

saveToDatabase(query, values)

setdown()

Writes session end and closes log file

setup()

does hardware initialization with(possibly updated) info in self.settingsDict Run by __init__, or can be run separately after editing the settingsDict

Returns

code

bool whether setup completed without errors

storeConfig(tag, configDict, source)

Stores configuration data, given as an IDtag, and dictionary for that tag, in some more permanent storage as a JSON text file, or a database or hd5 file, so it can be later retrieved by IDtag

writeToLogFile(tag, eventKind, eventDict, timeStamp, toShellOrFile=3)

The original standard text file method was 4 tab-separated columns, mouse tag, or 0 if no single tag was applicaple, unix time stamp, ISO formatted time, and event. Event could be anything. Now, every event has a kind, and every kind of event defines a dictionary. Main program calls writeToLogFile, as well as the Stimulator object For text based methods, event should be a dictionary for more complicated stimulator results, so an event can be more easily parsed during data analysis.

AHF_Datalogger.AHF_DataLogger_mysql module

class AHF_DataLogger_mysql.AHF_DataLogger_mysql(taskP, settingsDictP)

Bases: AHF_DataLogger.AHF_DataLogger

Simple text-based data logger modified from the original Auto Head Fix code makes a new text logfile for each day, saved in default data path.

Mouse data is stored in a specified folder, also as text files, one text file per mouse containing JSON formatted configuration and performance data. These files will opened and updated after each exit from the experimental tube, in case the program needs to be restarted The file name for each mouse contains RFID tag 0-padded to 13 spaces: AHF_mouse_1234567890123.jsn

REQUESTED VALUES: “exit” for exits as event “lever_pull” as event for lever data “positions” as key for the lever positions in the event_dictionary of “lever_pull”

PSEUDO_MUTEX = 0

The class field PSEUDO_MUTEX helps prevent print statements from different places in the code(main vs callbacks) from executing at the same time, which leads to garbled output. Unlike a real mutex, execution of the thread is not halted while waiting on the PSEUDO_MUTEX, hence the loops with calls to sleep to allow the other threads of execution to continue while waiting for the mutex to be free. Also, read/write to the PSEUDO_MUTEX is not atomic; one thread may read PSEUDO_MUTEX as 0, and set it to 1, but in the interval between reading and writing to PSEUDO_MUTEX,another thread may have read PSEUDO_MUTEX as 0 and both threads think they have the mutex

static about()

Returns a brief message describing your sub-class, used when asking user to pick a sub-class of this class

configGenerator(settings)

Each configuration file has config data for a single subject. This function loads data from all of them in turn, and returning each as a a tuple of(tagID, dictionary)

static config_user_get(starterDict={})

static method that querries user for settings, with default responses from starterDict, and returns starterDict with settings as edited by the user.

defaultCage = 'cage1'

defaultDatabase = 'AHF_laser_cage'

defaultHost = '142.103.107.236'

defaultPassword = '*******************'

defaultUser = 'slavePi'

getConfigData(tag, source)

returns a dictionary of data that was saved for this reference tag, in some permanent storage such as a JSON file Will be called when program is started, or restarted and settings need to be reloaded

getFromDatabase(query, values, remote)

getMice()

returns a list of mice that are in the dictionary/database

hardwareTest()

Tests functionality, gives user a chance to change settings. :Returns: bool – True if any settings have changed

localDatabase = 'raw_data'

localHost = 'localhost'

localPassword = '*********************'

localUser = 'pi'

makeLogFile()

Initiating database creation

newDay()

At the start of a new day, it was customary for the text-based data logging to start new text files, and to make a precis of the day’s results into a a separate text file for easy human reading. This “quickStats” file should contain info for each mouse with rewards, head fixes, or tasks, and other Stimulator specific data, which Stimulator object will provide for each mouse just call the Stimulator class functions for each mouse to get a dictionary of results

pingServers()

readFromLogFile(index)

Reads the log statement index lines prior to the current line. Returns the event and associated dictionary in a tuple.

retireMouse(tag, reason)

store information about a mouse retirement in a referenced file

saveNewMouse(tag, note, dictionary={})

store a new mouse entry in a referenced file

saveToDatabase(query, values, remote)

setdown()

Writes session end and closes log file

setup()

does hardware initialization with(possibly updated) info in self.settingsDict Run by __init__, or can be run separately after editing the settingsDict

Returns

code

bool whether setup completed without errors

storeConfig(tag, configDict, source)

Stores configuration data, given as an IDtag, and dictionary for that tag, in some more permanent storage as a JSON text file, or a database or hd5 file, so it can be later retrieved by IDtag

writeToLogFile(tag, eventKind, eventDict, timeStamp, toShellOrFile=3)

The original standard text file method was 4 tab-separated columns, mouse tag, or 0 if no single tag was applicaple, unix time stamp, ISO formatted time, and event. Event could be anything. Now, every event has a kind, and every kind of event defines a dictionary. Main program calls writeToLogFile, as well as the Stimulator object For text based methods, event should be a dictionary for more complicated stimulator results, so an event can be more easily parsed during data analysis.

AHF_Datalogger.AHF_DataLogger_text module

class AHF_DataLogger_text.AHF_DataLogger_text(taskP, settingsDictP)

Bases: AHF_DataLogger.AHF_DataLogger

Simple text-based data logger modified from the original Auto Head Fix code makes a new text logfile for each day, saved in default data path.

Mouse data is stored in a specified folder, also as text files, one text file per mouse containing JSON formatted configuration and performance data. These files will opened and updated after each exit from the experimental tube, in case the program needs to be restarted The file name for each mouse contains RFID tag 0-padded to 13 spaces: AHF_mouse_1234567890123.jsn

PSEUDO_MUTEX = 0

The class field PSEUDO_MUTEX helps prevent print statements from different places in the code(main vs callbacks) from executing at the same time, which leads to garbled output. Unlike a real mutex, execution of the thread is not halted while waiting on the PSEUDO_MUTEX, hence the loops with calls to sleep to allow the other threads of execution to continue while waiting for the mutex to be free. Also, read/write to the PSEUDO_MUTEX is not atomic; one thread may read PSEUDO_MUTEX as 0, and set it to 1, but in the interval between reading and writing to PSEUDO_MUTEX,another thread may have read PSEUDO_MUTEX as 0 and both threads think they have the mutex

static about()

Returns a brief message describing your sub-class, used when asking user to pick a sub-class of this class

configGenerator()

Each configuration file has config data for a single subject. This function loads data from all of them in turn, and returning each as a a tuple of(tagID, dictionary)

static config_user_get(starterDict={})

static method that querries user for settings, with default responses from starterDict, and returns starterDict with settings as edited by the user.

defaultCage = 'cage1'

defaultConfigPath = '/home/pi/Documents/MiceConfig/'

defaultDataPath = '/home/pi/Documents/'

getConfigData(tag)

returns saved dictionary for given tag

getMice()

returns a list of mice that are in the dictionary/database

hardwareTest()

Tests functionality, gives user a chance to change settings

makeLogFile()

open a new text log file for today, or open an exisiting text file with ‘a’ for append

makeQuickStatsFile(mice)

makes a quickStats file for today’s results.

QuickStats file contains daily totals of rewards and headFixes for each mouse

newDay()

At the start of a new day, it was customary for the text-based data logging to start new text files, and to make a precis of the day’s results into a a separate text file for easy human reading. This “quickStats” file should contain info for each mouse with rewards, head fixes, or tasks, and other Stimulator specific data, which Stimulator object will provide for each mouse just call the Stimulator class functions for each mouse to get a dictionary of results

readFromLogFile(tag, index)

Reads the log statement index lines prior to the current line. Returns the event and associated dictionary in a tuple.

retireMouse(tag, reason)

store information about a mouse retirement in a referenced file

saveNewMouse(tag, note, dictionary={})

store a new mouse entry in a referenced file

setDateStr()

Sets the string corresponding to todays date that is used when making files

setdown()

Writes session end and closes log file

setup()

copies settings, creates folders for data and stats, creates initial log file and writes start session

dataPath path to data folder

_|_

/ logPath statsPath subfolders within data folder

|

logFilePath statsFilePath paths to individual files within corresponding subfolders

storeConfig(tag, configDict, source='')

saves data to corresponding json text file, overwriting old file

writeToLogFile(tag, eventKind, eventDict, timeStamp, toShellOrFile=3)

Writes the time and type of each event to a text log file, and also to the shell

Format of the output string: tag time_epoch or datetime event The computer-parsable time_epoch is printed to the log file and user-friendly datetime is printed to the shell :param tag: the tag of mouse, usually from RFIDTagreader.globalTag :param eventKind: the type of event to be printed, entry, exit, reward, etc. :param eventDict: a dictionary containing data about the event(may be None if no associated data) returns: nothing

AHF_Datalogger.AHF_DataLogger_textMySql module

class AHF_DataLogger_textMySql.AHF_DataLogger_textMySql(taskP, settingsDictP)

Bases: AHF_DataLogger.AHF_DataLogger

Combination of the text data logger and mySQL data logger. Simply does both.

For Tim.

BUFFER_SIZE = 25

TO_FILE = 2

TO_SHELL = 1

static about()

Returns a brief message describing your sub-class, used when asking user to pick a sub-class of this class

configGenerator()

generates configuration data for each subject as(IDtag, dictionary) tuples from some kind of permanent storage such as a JSON file, or a database. Will be called when program is started, or restarted and settings need to be reloaded.

static config_user_get(starterDict={})

static method that querries user for settings, with default responses from starterDict, and returns starterDict with settings as edited by the user.

getConfigData(tag)

returns a dictionary of data that was saved for this reference tag, in some permanent storage such as a JSON file Will be called when program is started, or restarted and settings need to be reloaded

getMice()

returns a list of mice that are in the dictionary/database

hardwareTest()

Tests functionality, gives user a chance to change settings. :Returns: bool – True if any settings have changed

makeLogFile()

Makes or opens a text log file, or a datbase, or whatever else needs doing. Called once before entering main loop of program. DataLogger may make a new file every day in NewDay function, if desired

newDay()

At the start of a new day, it was customary for the text-based data logging to start new text files, and to make a precis of the day’s results into a a separate text file for easy human reading. This “quickStats” file should contain info for each mouse with rewards, head fixes, or tasks, and other Stimulator specific data, which Stimulator object will provide for each mouse just call the Stimulator class functions for each mouse to get a dictionary of results

readFromLogFile(index)

Reads the log statement index lines prior to the current line. Returns the event and associated dictionary in a tuple.

retireMouse(tag, reason)

store information about a mouse retirement in a referenced file

saveNewMouse(tag, note, dictionary)

store a new mouse entry in a referenced file

setdown()

oppposite of setup. Releases any hardware resouces. can be run before editing settings so GPIO pins can be reused, for example. This strategy should be used in hardwareTest method.

setup()

does hardware initialization with(possibly updated) info in self.settingsDict Run by __init__, or can be run separately after editing the settingsDict

Returns

code

bool whether setup completed without errors

storeConfig(tag, dictionary, source='')

Stores configuration data, given as an IDtag, and dictionary for that tag, in some more permanent storage as a JSON text file, or a database or hd5 file, so it can be later retrieved by IDtag

trackingDict = {}

useLocalSql = True

writeToLogFile(tag, eventKind, eventDict, timeStamp, toShellOrFile=3)

The original standard text file method was 4 tab-separated columns, mouse tag, or 0 if no single tag was applicaple, unix time stamp, ISO formatted time, and event. Event could be anything. Now, every event has a kind, and every kind of event defines a dictionary. Main program calls writeToLogFile, as well as the Stimulator object For text based methods, event should be a dictionary for more complicated stimulator results, so an event can be more easily parsed during data analysis.