Job Information
Federal Reserve System Senior Data Engineer in New York, New York
Company
Federal Reserve Bank of New York
Working at the Federal Reserve Bank of New York positions you at the center of the financial world with a unique perspective on national and international markets and economies. You will work in an environment with a diverse group of experienced professionals to foster and support the safety, soundness, and vitality of our economic and financial systems.
The Bank believes in work flexibility to balance the demands of work and life while also connecting and collaborating with our colleagues in person. Employees can expect to be in the office a couple of days per week as needed for meetings and team collaboration and should live within a commutable distance.
What we do:
The Enterprise Platforms and Capabilities (EPC) Technology Group function provides leadership to promote innovation and maturity of software development in support of the product and enterprise platform delivery teams for the NY Fed.
EPC is comprised of Enterprise Data Platforms and Enterprise Platforms divisions. The Enterprise Platforms division supports the Bank’s mission-oriented business groups and enabling functions by providing solution development on key Enterprise Platforms. Our enterprise platforms division enables key business capabilities including content management, constituent relationship management, workflow management, and governance and risk compliance. Our data platforms division enables delivery of enterprise data capabilities supporting our data strategy, including data management and analytics offerings. All our platform teams (also known as product squads) operate in an agile operating environment where DevSecOps principles promote collaboration across the business and various technical teams to deliver the highest business value. Our teams are focused on increasing speed to market while continuously improving quality, security, and resiliency.
Your role as a Senior Data Engineer:
Design, develop, monitor, and operate data pipelines and maintenance of existing data pipelines.
Collaborating with product owner and analysts to provide data-driven solutions and insights.
Implementing data transformations and aggregations for large datasets.
Ensuring data security and compliance with data governance policies.
Developing in languages like Pyspark and Python
Preprocess of structured and unstructured data
Analyze, transform, and create SQL queries.
Developing and optimizing Spark-based data processing applications on the AWS Databricks platform.
Troubleshooting and debugging data-related issues on the AWS Databricks platform.
Integrating Databricks with other data storage and processing systems.
What we are looking for:
Expertise in Apache Spark along with Spark streaming & Spark SQL.
Highly proficient with Apache Spark, Scala/ Python and PySpark.
Good understanding & hands-on experience Gitlab with CI/CD.
Hands-on experience in AWS Services like S3, Glue is required.
Working knowledge of AWS Databricks, Cluster Management.
Design, build, and test data pipelines and workflows using AWS Databricks.
Having the knowledge about Optimizing AWS Databricks Streaming jobs.
Have a deep understanding of AWS Cloud platforms and data engineering technologies.
Strong SQL skills to perform data analysis and understanding of source data.
Familiarity with NoSQL databases and JSON formats.
Experience in building ETL / data warehouse transformation processes
Strong knowledge of Data Management principles.
Good to have experience in analytical tool.
Good to have experience in Alteryx/Tableau.
Salary Range: $140000 - $230300 / year
We believe in transparency at the NY Fed. This salary range reflects a variety of skills and experiences candidates may bring to the job. We pay individuals along this range based on their unique backgrounds. Whether you’re stretching into the job or are a more seasoned candidate, we aim to pay competitively for your contributions .
Our Touchstone Behaviors —Communicate Authentically, Collaborate Inclusively, Drive Progress, Develop Others, and Take Ownership—help shape the culture of the Bank. They also provide a shared language for how we work together and achieve success, and they set clear expectations for leading with impact at every stage of your career with us. Learn more. (https://www.newyorkfed.org/careers/touchstone-behaviors)
Benefits:
Our organization offers benefits that are the best fit for you at every stage of your career:
Fully paid Pension plan and 401k with Generous Match
Comprehensive Insurance Plans (Medical, Dental and Vision including Flexible Spending Accounts and HSA)
Subsidized Public Transportation Program
Tuition Assistance Program
Onsite Fitness & Wellness Center
And more (https://www.newyorkfed.org/careers/benefits_health.html)
This position requires access to certain sensitive information and is therefore limited to individuals who are U.S. Citizens or lawful permanent resident aliens and who must undergo a separate background screening.
The New York Fed expects its employees to perform their duties with honesty, integrity, and impartiality, and without improper preferential treatment of any person. Learn more about our code of conduct and conflicts of interest rules. (https://www.newyorkfed.org/aboutthefed/ethics-conflicts-of-interest)
The Federal Reserve Bank of New York is committed to a diverse workforce and to providing equal employment opportunity to all persons without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, age, genetic information, disability, or military service.
We value accessibility for all candidates and are happy to provide an accommodation or assistance. Please email us at ny.leaves@ny.frb.org and we’ll be glad to help.
This is not necessarily an exhaustive list of all responsibilities, duties , performance standards or requirements , efforts, skills or working conditions associated with the job. While this is intended to be an accurate reflection of the current job, management reserves the right to revise the job or to require that other or different tasks be performed when circumstances change.
Full Time / Part Time
Full time
Regular / Temporary
Regular
Job Exempt (Yes / No)
Yes
Job Category
Information Technology
Work Shift
First (United States of America)
The Federal Reserve Banks believe that diversity and inclusion among our employees is critical to our success as an organization, and we seek to recruit, develop and retain the most talented people from a diverse candidate pool. The Federal Reserve Banks are committed to equal employment opportunity for employees and job applicants in compliance with applicable law and to an environment where employees are valued for their differences.
Privacy Notice (https://www.kansascityfed.org/documents/7797/Workday_Privacy_Notice.pdf)
OUR BANK has one of the most recognizable brands around the world. The Federal Reserve is the central bank of the United States—one of the world's most influential, trusted and prestigious financial organizations. The Federal Reserve is charged with the important mission of promoting a strong economy and a stable financial system and fulfills this responsibility by formulating national monetary policy, supervising and regulating banks and bank holding companies, and providing financial services for banks and the U.S. government.
OUR PEOPLE are diverse in background and ideas, which allows for ongoing creativity and innovation. Ultimately, they are the ones who push our high-performance, exchange-driven culture forward.
Why Our People Choose Us:
Our reputation precedes us
There will always be room for personal growth
Our people are first
You’ll find the right balance
Your responsibilities will be meaningful
We hope that you will be our future colleague.