Introduction
The Sentropy Detect API identifies abusive text posted by users in online communities. The API currently scores text strings for specific classes of abuse that can be used to monitor, categorize, investigate, and moderate user generated content.
Authorization
To authorize, use this code:
# Include the Authorization header with each request
curl "https://api.sentropy.io/v1" \
-H "Authorization: Bearer $TOKEN" \
[...]
import requests
requests.post(
"https://api.sentropy.io/v1",
headers={"Authorization": "Bearer $TOKEN"},
[...]
)
Make sure to replace
$TOKEN
with your API key.
Authorization to the API is performed using HTTP Bearer Auth. To obtain a bearer token, please reach out to apiaccess@sentropy.io.
Provide your bearer token on all API requests as a header, as in the following:
"Authorization: Bearer $TOKEN"
All API requests must be made over HTTPS. Requests made over HTTP or without a bearer token header will fail.
Get Abuse Classes
Single Message Api
This endpoint returns the probability of each class of abuse in the request.
curl "https://api.sentropy.io/v1" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"id": "4m8aw",
"author": "user_123",
"segment": "gaming_chat",
"text": "rl go to their offices and shoot everyone"
}'
import os
import requests
request_data = {
"id": "4m8aw",
"author": "user_123",
"segment": "gaming_chat",
"text": "rl go to their offices and shoot everyone"
}
TOKEN = os.environ["TOKEN"]
requests.post(
"https://api.sentropy.io/v1",
headers={"Authorization": f"Bearer {TOKEN}"},
json=request_data,
)
The above request will receive a JSON response payload similar to:
{
"id": "4m8aw",
"judgements": {
"IDENTITY_ATTACK": {
"score": 0.04475459083914757,
"is_severe": null
},
"IDENTITY_ATTACK/ABILITY": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/AGE": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/ETHNICITY": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/GENDER": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/POLITICAL_GROUP": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/RELIGION": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/SEXUAL_ORIENTATION": {
"score": null,
"is_severe": null
},
"INSULT": {
"score": 0.0002767886617220938,
"is_severe": null
},
"PHYSICAL_VIOLENCE": {
"score": 0.9987996816635132,
"is_severe": true
},
"PROFANITY": {
"score:": null,
"is_severe": null
},
"SELF_HARM": {
"score": 0.00015191955026239157,
"is_severe": null
},
"SEXUAL_AGGRESSION": {
"score": 3.632059815572575e-05,
"is_severe": null
},
"WHITE_SUPREMACIST_EXTREMISM": {
"score": 0.012745199725031853,
"is_severe": null
}
},
"segment": "gaming_chat",
"author": "user_123"
}
If an error occurs, the JSON response will be similar to:
{
"error": "'text' is a required property"
}
HTTP Request
POST https://api.sentropy.io/v1
Payload
id
(string)
An identifier for the text in this payload, unique among requests for a given authorized Sentropy Detect user.
author
(string)
An identifier for the author of the text in the payload.
segment
(string)
An identifier for the channel, room, "server", or other community that is under evaluation.
text
(string)
The text to be scored by Sentropy Detect for toxic content.
language
(string, optional)
A two-letter language code specifying the language. Currently supported values include
"en"
"es"
"pt"
"ru"
If omitted, language will be inferred from the contents of text
.
defend
(object, optional)
Additional fields used to display the message in Defend. See description of a defend object below.
Response Object
Attributes
id
(string)
This is the identifier provided on the request payload, unmodified.
author
(string)
This is the author identifier provided on the request payload, unmodified.
segment
(string)
This is the segment identifier provided on the request payload, unmodified.
judgements
(object of objects)
This is an object keyed by all class and subclass labels supported by the API for the request language, and valued by Judgement
objects.
error
(string)
This field will be present and populated with an error message if the request fails.
Judgement Object
Attributes
score
(float, nullable)
This is the estimated likelihood of the class label key to this Judgement
. It is valued on the closed interval [0.0, 1.0]
.
For some labels, the API may make no claim about the given payload. In these cases the score
is null
.
is_severe
(bool, nullable)
This is a binary flag indicating whether a message contains a severe form of toxicity. It takes on one of the three values: [true, false, null]
.
- true: the message contains a severe form of toxicity
- false: the message contains a mild form of toxicity
- null: either the message doesn't contain any form of toxicity, or severity is not defined for the class label
Defend Object
Attributes
permalink
(string, optional)
A URL to the message that is under evaluation. The permalink is shown in Defend and intended to provide moderators with message context.
Bulk Api
curl "https://api.sentropy.io/v1/bulk" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"messages": [
{
"id": "4m8aw",
"author": "user_123",
"segment": "gaming_chat",
"text": "rl go to their offices and shoot everyone"
},
{
"id": "fw837",
"author": "user_456",
"segment": "gaming_chat",
"text": "Im going to come to your house and kick you in the face"
}
]
}'
import os
import requests
request_data = {
"messages": [
{
"id": "4m8aw",
"author": "user_123",
"segment": "gaming_chat",
"text": "rl go to their offices and shoot everyone"
},
{
"id": "fw837",
"author": "user_456",
"segment": "gaming_chat",
"text": "Im going to come to your house and kick you in the face"
}
]
}
TOKEN = os.environ["TOKEN"]
requests.post(
"https://api.sentropy.io/v1/bulk",
headers={"Authorization": f"Bearer {TOKEN}"},
json=request_data,
)
This api can be used to submit up to 100 messages for judgement in a single POST call.
The individual messages in the array follow the same rules as the single message endpoint.
An error in one of the messages will not cause the entire request to fail.
The above request will receive a JSON response payload similar to:
{
"results": [
{
"id": "4m8aw",
"segment": "gaming_chat",
"author": "user_123",
"judgements": {
"IDENTITY_ATTACK": {
"score": 0.04475459083914757,
"is_severe": null
},
"IDENTITY_ATTACK/ABILITY": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/AGE": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/ETHNICITY": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/GENDER": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/POLITICAL_GROUP": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/RELIGION": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/SEXUAL_ORIENTATION": {
"score": null,
"is_severe": null
},
"INSULT": {
"score": 0.0002767886617220938,
"is_severe": null
},
"PHYSICAL_VIOLENCE": {
"score": 0.9987996816635132,
"is_severe": true
},
"SELF_HARM": {
"score": 0.00015191955026239157,
"is_severe": null
},
"SEXUAL_AGGRESSION": {
"score": 3.632059815572575e-05,
"is_severe": null
},
"WHITE_SUPREMACIST_EXTREMISM": {
"score": 0.012745199725031853,
"is_severe": null
}
}
},
{
"id": "fw837",
"author": "user_456",
"segment": "gaming_chat",
"judgements": {
"FINANCIAL_SCAM": {
"score": 0.00003109040699200705,
"is_severe": null
},
"IDENTITY_ATTACK": {
"score": 0.001138951163738966,
"is_severe": null
},
"IDENTITY_ATTACK/ABILITY": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/AGE": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/ETHNICITY": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/GENDER": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/POLITICAL_GROUP": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/RELIGION": {
"score": null,
"is_severe": null
},
"IDENTITY_ATTACK/SEXUAL_ORIENTATION": {
"score": null,
"is_severe": null
},
"INSULT": {
"score": 0.0001679484121268615,
"is_severe": null
},
"PHYSICAL_VIOLENCE": {
"score": 1.0000000447034836,
"is_severe": true
},
"PROFANITY": {
"score": null,
"is_severe": null
},
"SELF_HARM": {
"score": 0.000006043716894055251,
"is_severe": null
},
"SEXUAL_AGGRESSION": {
"score": 0.0023288412485271692,
"is_severe": null
},
"WHITE_SUPREMACIST_EXTREMISM": {
"score": 0.00000238938241636788,
"is_severe": null
}
}
}
]
}
HTTP Request
POST https://api.sentropy.io/v1/bulk
Payload
messages
(list of objects)
A list of messages with each message following the same structure as the payload for the single message endpoint.
Response Object
Attributes
results
(list of objects)
A list of objects where each object in the list follows the same structure as the single message response and has the same order as the request messages.
error
(string)
This field will be present and populated with an error message if the request fails.
Rate Limits
Sentropy Detect is rate limited. Authorization keys that exceed the rate limit will receive an HTTP 429 response.
The default rate limit is 3 messages per second, reset every 10 minutes. This comes out to a total of 1800 messages per 10 minutes which could be 1800 requests to the single message api or 18 requests to the bulk api with 100 messages each.
For information about your rate limit or to increase your rate limit, please reach out to apiaccess@sentropy.io.
Abuse Class Definitions
Identity Attack
The IDENTITY_ATTACK
class describes text that attacks people based on a shared identity, such as ethnicity, nationality, race, religion, gender, or sexual orientation, and meets any of the following criteria:
- Explicit use of slurs and other derogatory epithets referencing an identity group
- Violent threats or calls for harm directed at an identity group
- Calls for exclusion, domination or suppression of rights, directed at an identity group
- Dehumanization of an identity group, including comparisons to animals, insects, diseases or filth, generalizations involving physical unattractiveness, low intelligence, mental instability and/or moral deficiencies
- Expressions of superiority of one group over a protected or vulnerable group
- Admissions of hate and intolerance towards members of an identity group
- Denial of another’s identity, calls for conversion therapy, deadnaming
In addition to the general Identity Attack model, we provide pattern matchers for the different Identity Attack subclasses. We currently support the following Identity Attack subclasses:
Ability
The IDENTITY_ATTACK/ABILITY
subclass describes an identity attack directed at those having a physical or mental condition or disability.
The statement may contain explicit references to, or descriptions of, conditions or disabilities.
Age
The IDENTITY_ATTACK/AGE
subclass describes an identity attack directed at those belonging to a defined generation or any age group.
Ethnicity
The IDENTITY_ATTACK/ETHNICITY
subclass describes an identity attack directed at those belonging to an ethnicity, race, nationality, or geographical region.
Gender
The IDENTITY_ATTACK/GENDER
subclass describes an identity attack directed at those identifying with a specific gender.
Political Group
The IDENTITY_ATTACK/POLITICAL_GROUP
subclass describes an identity attack directed at members of a political group.
Religion
The IDENTITY_ATTACK/RELIGION
subclass describes an identity attack directed at members of a religious group.
Sexual Orientation
The IDENTITY_ATTACK/SEXUAL_ORIENTATION
subclass describes an identity attack directed at those identifying with a sexual orientation.
Insult
The INSULT
class contains insults directed at a person in a conversation.
An insult may refer to a person's physical traits (including race, sex, appearance), intelligence, personality, or behavior.
Insults may contain profanity or other offensive language, but it is not a prerequisite.
The subject of the insult should be someone involved in a conversation, not a celebrity, fictional character, or other third person. Self-directed insults are not considered positive instances of this class. Statements that use insults to describe concepts or non-human objects are not considered part of this class. Quoting or explaining an insult is not considered part of this class.
Physical Violence
The PHYSICAL_VIOLENCE
class describes text that meets any of the following criteria:
- Expresses desire to physically harm a person or group of people, including violent sexual acts
- Calls for the death of a person or group of people
- Encourages another person to commit self-injury, suicide or violence
Profanity
The PROFANITY
class returns a score of 1.0 if it detects the presence of curse words, sexually explicit terms, or
slurs pertaining to ethnicity, gender, race or sexual orientation, regardless of context.
Self Harm
The SELF_HARM
class describes text that meets any of the following criteria:
- Mentions the desire to deliberately harm one’s own body, such as via cutting, burning or dangerous behaviors
- Contains threats of suicide and/or conveys suicidal ideation
- Advice given to another individual regarding how to commit self harm and/or suicide.
User language referring only to depression, symptoms of depression, or depressed thoughts and feelings that are not tied to self-harm or suicide are not positives for this class. Recovery stories related to the topic of self harm and/or suicide are not considered part of this class.
Sexual Aggression
The SEXUAL_AGGRESSION
class describes text that contains obscene, graphic, sexual language directed at a person,
including threat of unwanted sexual acts. This includes at least one of the following:
- Graphic text describing a person (including oneself) that are sexual in nature
- Language that sexually degrades the subject
- Comments soliciting sexual behavior/images
- Descriptions of sexual behavior/fantasy/activity in the context of a non-mutually agreed upon sexual conversation
White Supremacist Extremism
WHITE_SUPREMACIST_EXTREMISM
is content seeking to revive and implement the ideology of white supremacists.
Ideologies encompassed by this class can include belief in any or all of the following:
white racial superiority, white cultural supremacy and nostalgia, white nationalism, eugenics,
Western traditional gender roles, racism, homophobia, xenophobia, anti-Semitism, Holocaust denial,
promotion of Jewish conspiracy theories,
idolization of Adolph Hitler, praise of Nazi policies or beliefs, use of Nazi symbols or slogans.
WHITE_SUPREMACIST_EXTREMISM
ideologies can be generalized into three categories which often overlap with each other:
Neo-Nazism, White Racial Supremacy, and White Cultural Supremacy.
Experimental Classes
Experimental Classes, listed below, are ones that are currently under development and that may lack robust domain and data coverage.
Financial Scam (Experimental)
The FINANCIAL_SCAM
class describes content that seeks to deceive individuals via deceptive or invasive techniques into providing funds or sensitive information. Financial scams are characterized by at least one of the following criteria:
- Attempts to trick users into sending money or sharing personal financial information (e.g. phishing)
- Promise of funds in return for a smaller initial payment via wire transfer, gift cards, or prepaid debit card (e.g. money-flipping)
- Offers promising cash or gifts, such as lottery scams
- Promises of debt relief or credit repair
Errors
The API uses the following error codes:
Error Code | Meaning | Action |
---|---|---|
422 | Unprocessable Entity -- The API has limitations that may cause it to return this error code with the error message “Unable to process request” These limitations are: | Do not retry, will continue to fail |
- Text fields containing individual sentences with more than ~40 words. | ||
- Text fields containing more than 200 sentences. | ||
- Text fields only containing non-alphanumeric characters | ||
400 | Bad Request -- the request is missing fields, has extra fields, or has incorrectly typed fields. | Fix request and retry |
401 | Unauthorized -- the request is made without an api token. | Provide API token and retry |
403 | Forbidden -- the API token is present but improperly formatted or not valid. | Double check API token and retry |
429 | Too Many Requests -- rate limit exceeded | Retry with an exponential backoff |