Disrupting Data Discovery featuring Lyft | San Francisco
About this event
Disrupting Data Discovery with Amundsen by Daniel Won, software engineer at Lyft:
Data scientists at Lyft spend approximately 30% of their time in the data discovery—answering questions such as Does this data exist? Who owns this data? What previous analysis exist? And can I trust this data? While data discovery is a prerequisite to delivering good analysis, it does not in itself bring value to the company. Reducing the time spent on data discovery enables data scientists to spend more time building models and visualizations.
Amundsen is an open-source tool built at Lyft that aims to solve the data discovery problem. We index and serve metadata about data resources in a simple and intuitive interface. A user can run a search that will return a list of results sorted by relevance and popularity. Currently, we index tables and people, with plans to index dashboards and teams as well. At Lyft, we have reduced the time spent in data discovery by 75% from baseline.