This project is mirrored from https://github.com/getsentry/snuba.
Pull mirroring updated .
- Jan 26, 2023
-
-
getsentry-bot authored
-
Rahul Kumar Saini authored
Removes the Python version of Generic Metrics Dataset, Entity, and Storages
-
- Jan 25, 2023
-
-
Enoch Tang authored
* add replays entity yaml * add test * add validate_data_model
-
Enoch Tang authored
* add profiles entity file * add test * minor changes
-
volokluev authored
Events entity generated with the [entity conversion script](https://github.com/getsentry/snuba/tree/python_2_yaml_entity_converter) * Added test functionality to compare translation mappers * Added test to verify join relationships * added test to verify subscription validators * The transactions entity was actually out of sync so I fixed it
-
volokluev authored
The sessions entity has a lot of complicated translation mappers which heavily make use of ColumnToFunction. We don't support ColumnToFunction because it would make a bad user experience and properly expressing it in config takes more thought than we care to give it right now. Therefore we are extracting all these complex mappers into their own file **Blast Radius** Only the session entity is affected which is deprecated. It is still used by single tenant and self hosted. There should be no code change here, this is refactoring
-
anthony sottile authored
-
- Jan 24, 2023
-
-
Enoch Tang authored
* Add triggered rate limiter name to querylog stats * remove print statement
-
- Jan 23, 2023
-
-
Dalitso Banda authored
* add migration to make query log match prod * Add indexes and change sample by and sorting key to match SaaS * properly format array modifiers * move primary key update to code migration * run update on all local nodes * rename migrations file to make it more descriptive * add comment * add comment * fix typo * use insert to op
-
Lyn Nagara authored
-
Lyn Nagara authored
"snuba-dead-letter-inserts" is not used anywhere, let's not unnecessarily define and create it.
-
volokluev authored
The discover Entity contains `FixedString` fields which do not exist in the configuration grammar right now * update the grammar * update the parser **Blast Radius** No existing entities should be affected **Feedback Desired** Is there a better place to put the test that I don't know about?
-
Evan Hicks authored
GroupedMessage and GroupAssignee entities had YAML versions created and added to their existing datasets. The two entities are now being loaded from config instead of Python. `required_time_column` can now be set to `null`. The field is still required in the schema since that is the main case. These entities are a special case. There is now a test in the entity loader that is similar to the storages test that compares entities to their Python version. As we migrate entities we should update that test. Once our migration is done we can delete that test entirely.
-
MeredithAnya authored
* ref(migrations): Simplfy policies and status checks
-
Nikhar Saxena authored
Since there is no TTL on the outcomes dataset, the storage size can grow unbounded. This PR adds a TTL of 30 days for the raw tables and 90 days for the aggregate tables
-
MeredithAnya authored
* ref(copy-tables): fix regex
-
Tony Xiao authored
This adds a new profile id column to the transactions table as per https://github.com/getsentry/rfcs/blob/main/text/0047-introduce-profile-context.md
-
Evan Hicks authored
Generated using the pyentity_2_yaml.py script. Changes: The outcomes entity is using the `EntityContainsColumnsValidator` at the warn level, so the `validate_data_model` is now part of the entity schema to define that.
-
- Jan 20, 2023
-
-
Lyn Nagara authored
This allows us to better route Sentry errors to relevant teams based on tags. The storage name is also tagged on the Snuba consumer so we can filter by consumers for monitoring.
-
Lyn Nagara authored
This is checked https://github.com/getsentry/snuba/blob/e50b6ab0b02af07bba348e56f611794aa363954f/snuba/subscriptions/combined_scheduler_executor.py#L78-L79 so we know it is the same.
-
Lyn Nagara authored
Remove the ability to override the scheduling mode on the combined scheduler/executor. This hack previously existed for single tenant's `transactions` topic. Since the topic was semantically partitioned but the mode was global, empty partitions would cause the entire pipeline to stop on a regular basis. This combination (semantic partitioning with subscriptions in global mode) no longer exists so this is not needed.
-
Enoch Tang authored
* add timing metrics for table_concurrent * rename metric to table_concurrent_hist
-
Lyn Nagara authored
-
volokluev authored
Co-authored-by: Volo Kluev <volo.kluev@sentry.io>
-
- Jan 19, 2023
-
-
Dalitso Banda authored
* make errors schema match SaaS * add tests for and use modify to reset settings to default * add hashes cols to error_dist_ro
-
volokluev authored
* Converting sessions and errors entities is not possible until the storage selectors can be referenced from config. This only works now because only the default ones are used in config
-
Lyn Nagara authored
The parallel collect step is too complex, and offers little more than what can be achieved with Reduce chained to RunTaskInThreads. As a follow up, Collect/ParallelCollect will be completely removed from Arroyo. The collect strategy is too complicated and we have introduced more intuitive ways to do batching now (via the Batch and Reduce steps).
-
Rahul Kumar Saini authored
-
volokluev authored
* fix ColumnToIPAddress mapper needing unnecessary parameters * update config * update events and discover entities Co-authored-by: Vlad Kluev <vlad.kluev@sentry.io>
-
Riya Chakraborty authored
* add help comment * add strategy test to consumer builder * add break comments * more debugging steps * Commit call verified * Add clickhouse row count test * Remove comments * change sleep time * comment out commit call check * retry commit check * change bootstrap servers line
-
Rahul Kumar Saini authored
* errors_ro * Add errors_ro to tests * mandatory conditions
-
Ethan Smith authored
This bumps the action ref to getsentry/action-self-hosted-e2e-tests@20b5170, which introduces sentry-cli, a new dependency of our integration tests.
-
- Jan 18, 2023
-
-
Dan Fuller authored
When we have an error here (for example, due to a dataset not existing) we end up with an obscure connection error in sentry. Explicitly raising an `InternalServerError` causes a 500 to be returned.
-
Rahul Kumar Saini authored
Discover dataset has a unique entity selection process This blocks conversion to config This logic shouldn't be generalized and should remain specific to discover, so I've moved it into the parser and freed up the dataset to be converted
-
Colton Allen authored
* Add error and session sample rate columns * Reverse column ordering
-
Enoch Tang authored
### Summary Currently, we are not able to move the `sessions` entity into configuration. This is because sessions contains different translation mappers for each of its selectable storages. This PR is responsible for reformatting how storages and translation mappers are defined in configuration. Entities now specify a `Sequence[EntityStorageConnection]`. In addition, a `SimpleQueryStorageSelector` was added to support this new format of storages in configuration. ### Before State Entity configuration: ``` readable_storage: generic_metrics_distributions writable_storage: generic_metrics_distributions_raw translation_mappers: [x,y,z] ``` * Storages were "implicitly" selected in code. For example, when there are two or more storages defined in an entity, only one single storage (usually the materialized_storage) is passed into the query plan builder. ### After State * Entity configuration: ``` storages: - storage: generic_metrics_distributions translation_mappers: [x,y,z] - storage: generic_metrics_distributions_raw is_writable: true translation_mappers: [x,y,z] ``` * All storages defined in configuration are passed into the query plan builder. It is the job of the query storage selector to determine which storage to use. As a result, all storages need to be selected explicitly in configuration. The addition of `SimpleQueryStorageSelector` helps with that. ### Blast Radius The configuration of all entities have been changed to use this format. ### Testing * Entity definitions and pluggable entity tests have been updated to use new format * A new `SimpleQueryStorageSelector` added.
-
- Jan 17, 2023
-
-
Rahul Kumar Saini authored
* add cdc datasets * rename * fix test
-
Rahul Kumar Saini authored
* json schema and storage builder * finish up * Add cdc storages * folder restructure to reflect dataset structure * rename
-
Colton Allen authored
-
Rahul Kumar Saini authored
Previously only Readable and Writable storages were supported This doesn't work for the CDC storages Introduced cdc_storage config type and refactored storage building to support the new type
-