Software Engineering Daily
Spela

Facebook Data Infrastructure with Dhruba Borthakur

Software Engineering Daily

00:00

Facebook Data Infrastructure with Dhruba Borthakur

Software Engineering Daily

Technical interviews about software topics.

Software Engineering Daily

Facebook generates high volumes of data at a rapid pace. Dhruba Borthakur joined Facebook in 2008 to work on data infrastructure. His early projects at Facebook were around Hadoop, the distributed file system and MapReduce computation platform that laid the foundation for the “big data” movement.  At the time, Facebook was generating as much data The post Facebook Data Infrastructure with Dhruba Borthakur appeared first on Software Engineering Daily.

Published

Play Episode

Related episodes Software Engineering Daily

Software Engineering Daily

Facebook Engineering Culture with Raylene Yung
(NaN)
Facebook moves fast because of vision, collaboration, and trust. The fast pace of development is enabled by constantly improving infrastructure and a sense of unity throughout the company.  In Facebook’s early days, there was an emphasis on rapidly deploying new code to drive constant improvement and experimentation.  Product quality was maintained by engineers closely checking The post Facebook Engineering Culture with Raylene Yung appeared first on Software Engineering Daily....

en

Software Engineering Daily

Facebook PHP with Keith Adams
(NaN)
Facebook was built using PHP, a programming language that was used widely in the late 90s and early 2000s. PHP allows developers to get web applications built quickly and easily, although PHP has a reputation for being difficult to scale.  In the early days of Facebook, the company was scaling rapidly on every dimension.  New The post Facebook PHP with Keith Adams appeared first on Software Engineering Daily....

en

Software Engineering Daily

Meltano: Data Engineering Lifecycle with Danielle Morrill
(NaN)
Data engineering allows a company to take advantage of the large quantities of data that the company has generated. In many companies, new data has been produced rapidly for many years, but the company has not been able to take full advantage of it.  Creating large data sets does not provide immediate value for a The post Meltano: Data Engineering Lifecycle with Danielle Morrill appeared first on Software Engineering Daily....

en

Software Engineering Daily

Software Media with Charles Max Wood
(NaN)
Software engineers have a wide variety of media to choose from, including podcasts, blogs, YouTube videos, conferences. The amount of software engineering media that is available is growing and accelerating. Eight years ago, there were not as many options for information about software. Charles Max Wood founded Devchat.tv to create a network of podcasts and The post Software Media with Charles Max Wood appeared first on Software Engineering Daily....

en

Software Engineering Daily

gVisor Container Isolation with Michael Pratt and Yoshi Tamura
(NaN)
Software applications running within a host operating system need to be isolated. Isolation prevents security vulnerabilities, such as one application accessing the memory of another. In modern cloud environments, a single physical host might be running multiple virtual machines on top of a hypervisor. Those virtual machines might be divided up into containers. The different The post gVisor Container Isolation with Michael Pratt and Yoshi Tamura appeared first on Software Engineering Daily....

en

Software Engineering Daily

Software Free Radicals with Lior Kanfi
(NaN)
Creating a software company has never been easier. Software engineers are increasingly seeing entrepreneurship as a viable career path. But the path to being an independent software developer is not always clear.  Most engineers spend some of their career working at a software company. Even an engineer who intends to build a company someday can The post Software Free Radicals with Lior Kanfi appeared first on Software Engineering Daily....

en

Software Engineering Daily

Edge Storage with Steve Klabnik
(NaN)
Edge computing allows for faster data access and computation. When your client application makes a request, that request might be routed to the edge. Edge servers are more numerous and more widely distributed than normal data centers, but an edge server might not have all of the data or the complete application logic for the The post Edge Storage with Steve Klabnik appeared first on Software Engineering Daily....

en

Software Engineering Daily

You Are Not A Commodity (Keynote at Tikal Full Stack Tech Radar Day)
(NaN)
Today’s episode is a keynote I gave at Full Stack Tech Radar Day in Tel Aviv. The talk is called “You Are Not a Commodity”. This talk is also available as a YouTube video. The slides can be accessed here. The world of commodity engineering is coming to an end. Developers are becoming more productive, The post You Are Not A Commodity (Keynote at Tikal Full Stack Tech Radar Day) appeared first on Software Engineering Daily....

en

Software Engineering Daily

ListenNotes: Podcast Search Engine with Wenbin Fang
(NaN)
ListenNotes allows users to find podcasts by categories, popularity, and search queries. ListenNotes is not a podcast client that you download onto your phone, but it allows users to build playlists. These playlists are automatically published to an RSS feed, so that users can curate a playlist on desktop and subscribe to that playlist from The post ListenNotes: Podcast Search Engine with Wenbin Fang appeared first on Software Engineering Daily....

en

Software Engineering Daily

Permissionless Innovation with Joseph Jacks
(NaN)
Open source software allows developers to take code from the Internet and modify it for their own use. Open source has allowed innovation to occur on a massive scale. Today, open source software powers our consumer client applications and our backend cloud server infrastructure.  Linux powers single node operating systems and Kubernetes is the foundation The post Permissionless Innovation with Joseph Jacks appeared first on Software Engineering Daily....

en

Software Engineering Daily

Google Earth WebAssembly with Jordon Mears
(NaN)
Google Earth allows users to explore the imagery of the real world. Imagery for Google Earth is taken from satellites, cars equipped with cameras, and other sources. Google Earth renders a data intensive 3-D model of the world on a client application such as a desktop browser or virtual reality system.  WebAssembly is a runtime The post Google Earth WebAssembly with Jordon Mears appeared first on Software Engineering Daily....

en

Software Engineering Daily

FoundationDB with Ryan Worl
(NaN)
FoundationDB is a multi-model distributed key-value store. It is fully ACID compliant and horizontally scalable. FoundationDB is not usually used directly by an application developer–FoundationDB is a foundational building block for higher level distributed systems such as the metadata store for data warehousing tool Snowflake. Ryan Worl is a software engineer who specializes in FoundationDB. The post FoundationDB with Ryan Worl appeared first on Software Engineering Daily....

en

Software Engineering Daily

FindCollabs Hackathon Winners: Kitspace and Rivaly
(NaN)
FindCollabs is a platform for finding collaborators and building projects. Three months ago we had our first hackathon, with lots of projects being created and collaborated on. In an earlier episode, we showcased the first place winner ARhythm.  Today’s show features two more interviews with winners from the first FindCollabs hackathon. Kitspace is an open The post FindCollabs Hackathon Winners: Kitspace and Rivaly appeared first on Software Engineering Daily....

en

Software Engineering Daily

Citus Data: Founding to Acquisition with Umur Cubukcu
(NaN)
A new database company needs to solve numerous problems in order to succeed. There are already lots of existing database companies, so a new company needs to find a way to strongly differentiate itself. Databases are core infrastructure, so a new database company must earn trust with its customers. A database is a complicated distributed The post Citus Data: Founding to Acquisition with Umur Cubukcu appeared first on Software Engineering Daily....

en

Software Engineering Daily

Lambda School Education with Andrew Madsen
(NaN)
Programmers are in high demand, and software engineering is a career path that is fun, creative, and lucrative. There are many people who want to transition into a career in software and are looking for the right path toward writing code. The traditional college computer science curriculum teaches some software engineering skills, but the time The post Lambda School Education with Andrew Madsen appeared first on Software Engineering Daily....

en

Software Engineering Daily

Afresh: Grocery Store Software with Volodymyr Kuleshov
(NaN)
A grocery store contains fruit, vegetables, meat, bread, and other items that can expire. In order to keep these items in stock, the store must be aware of how much food has been sold and what has gone bad. When a food item is low in stock, the store needs to order more of that The post Afresh: Grocery Store Software with Volodymyr Kuleshov appeared first on Software Engineering Daily....

en

Software Engineering Daily

Airflow in Practice with Chaim Turkel
(NaN)
Apache Airflow is a system for scheduling and monitoring workflows for data engineering. Airflow can be used to schedule ETL jobs, machine learning work, and script execution. Airflow also gives a developer a high level view into the graph of dependencies for their data pipelines. Chaim Turkel is a backend data architect at Tikal. He The post Airflow in Practice with Chaim Turkel appeared first on Software Engineering Daily....

en

Software Engineering Daily

Project Management with Kurt Schrader
(NaN)
Software projects are organized and planned using project management software. Examples of project management software include JIRA, Trello, and Asana. There are hundreds of tools for managing a software project because there are infinite ways that a project could be managed. Google Docs changed project management by allowing documents to be easier to share and The post Project Management with Kurt Schrader appeared first on Software Engineering Daily....

en

Software Engineering Daily

Niantic Real World with Paul Franceus
(NaN)
Niantic is the company behind Pokemon Go, an augmented reality game where users walk around in the real world and catch Pokemon which appear on their screen. The idea for augmented reality has existed for a long time. But the technology to bring augmented reality to the mass market has appeared only recently. Improved mobile The post Niantic Real World with Paul Franceus appeared first on Software Engineering Daily....

en

Software Engineering Daily

WebAssembly Compilation with Till Schneidereit
(NaN)
WebAssembly allows for web-based execution of languages other than JavaScript. Programs written in Rust or C++ can be compiled down to WebAssembly and shipped over the browser for on-the-fly execution in a safe, memory controlled environment. WebAssembly has been in development for more than two years, and is still an immature ecosystem because building the The post WebAssembly Compilation with Till Schneidereit appeared first on Software Engineering Daily....

en

Software Engineering Daily

Infrastructure Wars with Sheng Liang
(NaN)
Sheng Liang was the lead developer on the original Java Virtual Machine. Today he works as the CEO of Rancher Labs, a company building a platform on top of Kubernetes. Sheng joins the show to discuss his experiences in the technology industry. The container orchestration wars had many victims. The competing standards for how an The post Infrastructure Wars with Sheng Liang appeared first on Software Engineering Daily....

en

Software Engineering Daily

Kubernetes Operators with Rob Szumski
(NaN)
Kubernetes has made distributed systems easier to deploy and manage. As Kubernetes has become reliable, engineers have started to look for higher level abstractions we can define on top of Kubernetes. An operator is a method of packaging, deploying, and managing a Kubernetes application. Operators are useful for spinning up distributed systems such as Kafka, The post Kubernetes Operators with Rob Szumski appeared first on Software Engineering Daily....

en

Software Engineering Daily

Render: High Level Cloud with Anurag Goel
(NaN)
Cloud computing was popularized in 2006 with the launch of Amazon Web Services. AWS allowed developers to use remote server infrastructure with a simple set of APIs. But even with AWS, it was still not simple to deploy and manage a web application. In 2007, Heroku launched a platform built on top of AWS. Heroku The post Render: High Level Cloud with Anurag Goel appeared first on Software Engineering Daily....

en

Software Engineering Daily

Elegant Puzzle with Will Larson
(NaN)
Software engineering is an art and a science. To manage engineers is to manage artists and scientists. Software companies build practical tools like payment systems, messaging products, and search engines. Software tools are the underpinnings of our modern lives. You might expect this core infrastructure which modern humans rely on to have been constructed with The post Elegant Puzzle with Will Larson appeared first on Software Engineering Daily....

en

Software Engineering Daily

Stripe Machine Learning Infrastructure with Rob Story and Kelley Rivoire
(NaN)
Machine learning allows software to improve as that software consumes more data. Machine learning is a tool that every software engineer wants to be able to use. Because machine learning is so broadly applicable, software companies want to make the tools more accessible to the developers across the organization. There are many steps that an The post Stripe Machine Learning Infrastructure with Rob Story and Kelley Rivoire appeared first on Software Engineering Daily....

en