Event-based Data Capture refers to the real-time reception of data as it occurs. The specific configuration for enabling this capability may differ across platforms; however, the reported objects will generally contain similar content in both platforms.

Data Objects

These are the different data objects.

SessionUpdate

Property Name	Unit	Type	Description	Platform
language	name	string	Current language that the user is typing in. ISO 639-1 - ISO 3166-1 representation. Ex: en-US	Both
languageVersion	name	string	Current version for the dictionary language that we are using.	Both
layout	name	string	Current layout used for the keyboard, which depend on the language and user preferences. Example: QWERTY, AZERTY, ..	Both
textFieldType	name	enum	Current Textfield where the user is writting.	Both
appContext	name	string	Current package name of the app where the keyboard is shown. From iOS16 this is not possible, so it might be empty.	Both
schemaVersion	name	string	JSON schema version the captured data complies to following semver standard	Both
timeZone	hours	number	Current Timezone referenced to GMT	Both
startTimestamp	ms	number	Timestamp for session start time	Both
endTimestamp	ms	number	Timestamp for session end time. If the sesssion is still active, this value will be zero.	Both
keyboardAreaWidth	pixels	number	Keyboard width positioned inside the screen in pixels	Both
keyboardAreaHeight	pixels	number	Keyboard height positioned inside the screen in pixels	Both
screenWidthMm	mm	number	Screen width of the phone/tablet in mm	Both
screenHeightMm	mm	number	Screen height of the phone/tablet in mm	Both
screenWidthPx	pixels	number	Screen width of the phone/table in pixels, total number of pixels	Both
screenHeightPx	pixels	number	Screen height of the phone/table in pixels, total number of pixels	Both
sessionText	text	string	Transcription of the final text input content by the end of the session.	Both

TextFieldType

Current Textfield where the user wrote. There might be different values depending on the platform.

This object contains the following values for Android:

Text: Regular Text field

Password: Password text field

URL: URL text field

Email_Address: Email text field

Numbers: Numbers text field

Twitter: Twitter text field, used in Twitter/Slack / …

Websearch: Websearch textfield

User_AC_OFF: Regular text field without Autocorrection

No_Suggestions: Regular text field without suggestions

Twitter_AC_OFF: Twitter text field without AC

Phone: Phone text field for adding a phone number

Undefined: None of the above

This object contains the following values for iOS:

regularText: Regular Text field

url: URL text field

emailAddress: Email text field

numbers: Numbers text field

twitter: Twitter text field, used in Twitter/Slack / …

websearch: Websearch textfield

undefined: None of the above

SessionText

sessionText is the accumulated text collected during the session. Value will be filled/available upon session’s termination (last sessionUpdate event within a session).

We have special characters to distinguish different behaviours:

\n When the end-user presses enter we mark this as \n in the reporting

<probable>probable text</probable> We mark text as <probable> when the keyboard is not able to distinguish if that text is present or not. There are two specific cases when this behaviour can be experienced:
1. When the whole text in the textfield is cut.
2. When the whole text in the textfield is selected and the text is committed, e.g. using send button in the Messages app.
This is a new parameter introduced in these versions: iOS 4.14.2 and Android 4.4.21


`\n`	When the end-user presses enter we mark this as `\n` in the reporting
`<probable>probable text</probable>`	We mark text as `<probable>` when the keyboard is not able to distinguish if that text is present or not. There are two specific cases when this behaviour can be experienced: 1. When the whole text in the textfield is `cut`. 2. When the whole text in the textfield is selected and the text is `committed`, e.g. using `send` button in the Messages app. This is a new parameter introduced in these versions: `iOS` 4.14.2 and `Android` 4.4.21

KeyStroke

The keystroke object is generated when the end-user taps any key.

Property Name	Unit	Type	Description	Platform
pressTime	ms	number	Time relative to the startUnixTime reported in the SessionUpdate object.	Both
releaseTime	ms	number	Time relative to the startUnixTime reported in the SessionUpdate object.	Both
keyPressBegin	points	(x,y)	-	Both
keyPressEnd	points	(x,y)	-	Both
keyCenter	pixels	(x,y)	Center of the key that we are pressing reference to the screen area in pixels	Both
keyRectangle	pixels	(x,y,width,height)	Dimension of the key that we are pressing reference to the screen area in pixels	Both
keyType	-		Key type indicating character family	Both
keyCode	number	number	Keystroke character equivalent UTF8 Unicode in Decimal. First characters its equivalent to ASCII32.	Both
keyArea	-	number	Description of the area where the user has pressed. Values from 0 to 5	Both
keyText	string	text	Current text pressed, which could be a character, an emoji.	Both

keyTypes

Key type indicating character family.

This object contains the following values:

alphanumeric: Alphanumeric

punctuation: Symbols and punctuation

spacebar: Space bar

backspace: Backspace

enter: Enter

modifierKey: Modifier key

shift: Shift

autoShift: Automatic shift

autoToggleNumbers: Automatic numbers

toggleEmojis: Emoji button / keyboard from emoji (Note: This value was “em” in the sdk versions iOS 3.X.X and Android 3.X.X)

prediction: Prediction from the topbar

suggestion: Suggestion from the topbar

other: Others

keyCode

Keystroke character equivalent UTF8 Unicode in Decimal. First characters its equivalent to ASCII32.

Note:

Number 000 makes reference to actions.

Number 900 makes reference to emoji. The transcription of this emoji it is in the array of emojis, by order.

Number 901 makes reference to a prediction.

keyArea

Description of the area where the user has pressed. This is a keyArea enum:

This object contains the following values:

undefined

QWASZ

ERDFXC

TYGV

UIHJBN

OPKLM

suggestionBar (Note: Versions previous of iOS 4.8.4 and Android 4.2.3 was “9”)

Word

Information related to each of the words.

Property Name	Type	Description
originalWord	string	Original word written (in case of AC, Prediction or swipe)
word	string	Word recorded
isAutoCorrectedWord	number	It comes from an autocorrection if 1.
isPredictedWord	number	It comes from a prediction if 1.
isSwipeWord	number	It comes from a swipe typing if 1.
autocorrectionType	enum	Type of autocorrection in case of autocorrection true.
autocorrectionValue	number	Value related to autocorrectionType
timestamp	number	Time from the start of the session
uniP	number	Unigram log probability of this word

AutocorrectionType

Values:
missedSpace – in this case value is number of words splitted into
misstypedSpace – Miss an space
eliminated – Characters deleted
transpositioning – Value of characters changed
missingTap – Value of missed taps in the word
none – empty in case of no type

Swipe

Collection of all data related to a trace on top of the keyboard to detect a word.

Property Name	Type	Description
layout	string	Specific keyboard layout for swipe. Example: QWERTY, AZERTY, ..
context	string	Previous word of the swipe
word	string	Final written word
selectedSuggestion	string	This is empty unless the user presses one of the suggestions of the topbar.
feature	`<Array> of <`Features`>`	-
candidates	`<Array> of <`Candidates`>`	Array of candidates proposed based on the features.
timestamp	number	Time from the start of the session.

Features

Property Name	Type	Description
point	(x,y)	Point for each of the features
label	string	Letter detected
timestamp	number	Path point timestamp from start of the session

Candidates

Property Name	Type	Description
candidate	string	Word
shapeProb	number	Shape log probability
finalProb	number	Final log probability

Delete

Delete object with the data related to the delete action.

Property Name	Type	Description
word	string	Word being delete before applying the delete action
deleteCharacter	string	Character to delete
typeDelete	number	Type of delete `deleteCharacter` – Delete a Character `deleteSpace` – Delete a Space `undoAC` – Delete action that undo previous AC
timestamp	number	Time from the start of the session

KeyPlane

Key plane object is the representation of all the different keys drawn in the screen.

Property Name	Object	Description
keyPlaneId	KeyboardPlaneType
keys	Array of Key

KeyboardPlaneType

It holds all different type of keyboard planes:

qwertyLower / qwertyUpper / qwertyAC_OffLower / qwertyAC_OffUpper
numbers / numberPad / numberPadSymbols / number_iOS / numbersMini
symbols / symbols_iOS
shift1 / shift2 / shift3
phonePad / phonePadSymbols
emojis
twitterLower / twitterUpper / urlLower / urlUpper / emailLower / emailUpper
temp
invalid

Key

Property Name	Type	Description
label	string	Label associated with key if any
center	(x,y)	Center of the key
rectangle	(x,y,width,height)	Rectangle space of the whole key.

StressMonitor

When we have the capability of stress enabled we will be able to receive this object too.

Property Name	Type	Description
stress	Stress	Which level of stress is detecting the system.
regime	Regime	It represents the current status of the system: training or not.

Stress

Property Name	Type	Description
Status	enum	Values: stressDetected / stressNotDetected / Undefined

Regime

Property Name	Type	Description
regime	enum	Returns if the system is in `training` mode or `inference mode.