Data Engineer, Rust

Edge & Node

Edge & Node

Software Engineering, Data Science
North America · South America · Remote
Posted on Thursday, February 22, 2024

Edge & Node stands as the revolutionary vanguard of web3, a vision of a world powered by individual autonomy, shared self-sovereignty and limitless collaboration. Established by trailblazers behind The Graph, we’re on a mission to make The Graph the internet’s unbreakable foundation of open data. Edge & Node invited and standardized subgraphs across the industry, solidifying The Graph as the definitive way to organize and access blockchain data. Utilizing a deep expertise in developing open-source software, tooling, and protocols, we empower builders and entrepreneurs to bring unstoppable applications to life with revolutionary digital infrastructure.

Edge & Node acts on a set of unwavering principles that guide our journey in shaping the future. We champion a decentralized internet - free from concentrated power - where collective consensus aligns what is accepted as truth, rather than authoritative dictation. Our commitment to censorship resistance reinforces our vision of an unyielding information age free from the grasp of a single entity. By building for open-source, we challenge the stagnant landscape of web2, recognizing that true innovation thieves in transparency and collaboration. We imagine a permissionless future where the shackles imposed by central gatekeepers are not only removed, but relegated to the dustbin of a bygone era. And, at the foundation of it all, our trust shifts from malevolent middlemen to trustless systems, leveraging smart contracts to eliminate the age-old vulnerabilities of misplaced trust.

The Graph Network Engineering team is focused on building and maintaining core software components that make The Graph work. We are especially focused on graph-node, an Open Source Rust project that provides the core indexing capabilities of The Graph.

We are looking for a Data Engineer with deep understanding of relational databases and modern data stacks, and a desire to code in Rust. A Rust Data Engineer at E&N will not just use Rust data tools, but build our general purpose blockchain data systems and also contribute to our dependencies.

What You’ll Be Doing

  • Working within the Graph Node team to improve Graph Node’s capabilities as a data processing and querying engine
  • Familiarizing yourself with the large and complex Rust codebase that is Graph Node
  • Using subgraphs to understand both the specification implemented by Graph Node as well as the developer experience of the builders who are our users.
  • Develop an understanding of existing code and design choices, and then analyze how they can be experimented with and improved upon
  • Setting up performance tests, doing quantitative assessments of any proposals and changes you make, and monitoring how the changes ultimately behave when rolled out
  • Experimentally validating, and if necessary, falsifying your own ideas.
  • Reviewing pull requests of your colleagues and taking responsibility for the reviewed code as if it were your own
  • Documenting important aspects of the software while understanding that Graph Node has a large community of independent operators that need to understand how to run, configure, and monitor Graph Node

What We Expect

  • You’ve contributed more than 10,000 lines of code to a production software product
  • Thinking about data in SQL terms comes naturally for you
  • You have an understanding of some of the theory behind data systems, such as relational algebra, and an understanding of fundamental DB data structures and data formats, such as BTrees, LSM Trees, and compression formats
  • You have an interest in higher-level query languages such GraphQL
  • You have an interest in federated data and federated query frameworks
  • You understand how small design choices can have large effects in big data systems
  • You are comfortable working with an existing system and codebase and making fundamental improvements to it while being mindful of backwards compatibility
  • You understand tracing and debugging requirements of complex software
  • You have an understanding of the academia and theory that backs the software libraries you use but also understand the pragmatic necessity of shipping usable software quickly

About the Graph

The Graph is the indexing and query layer of web3. The Graph Network’s self service experience for developers launched in July 2021. Developers build and publish open APIs, called subgraphs, that applications can query using GraphQL. The Graph supports indexing data from multiple different networks including Ethereum, NEAR, Arbitrium, Optimism, Polygon, Avalanche, Celo, Fantom, Moonbeam, IPFS, and PoA with more networks coming soon. To date, tens-of-thousands of subgraphs have been deployed on the hosted service, and now subgraphs can be deployed directly on the network. Over 28,000 developers have built subgraphs for applications such as Uniswap, Synthetix, KnownOrigin, Art Blocks, Balancer, Livepeer, DAOstack, Audius, Decentraland, and many others.

If you are a developer building an application or web3 application, you can use subgraphs for indexing and querying data from blockchains. The Graph allows applications to efficiently and performantly present data in a UI and allows other developers to use your subgraph too! You can deploy a subgraph to the network using the newly launched Subgraph Studio or query existing subgraphs that are in the Graph Explorer. The Graph would love to welcome you to be Indexers, Curators and/or Delegators on The Graph’s mainnet. Join The Graph community by introducing yourself in The Graph Discord for technical discussions, join The Graph’s Telegram chat, and follow The Graph on Twitter, LinkedIn, Instagram, Facebook, Reddit, and Medium! The Graph’s developers and members of the community are always eager to chat with you, and The Graph ecosystem has a growing community of developers who support each other.

The Graph Foundation oversees The Graph Network. The Graph Foundation is overseen by the Technical Council. Edge & Node, StreamingFast, Messari, Semiotic and The Guild are five of the many organizations within The Graph ecosystem.