Project Title:
Real-time Price Aggregation and Normalization for US Structured Finance Securities
Project Objective:
To enhance the efficiency and accuracy of pricing US structured finance securities by automating the process of collecting, cleaning, and normalizing market price data from multiple sources.
Technical Approach:
1. Data Ingestion
- Developed Python scripts to extract real-time price data from various sources, including Bloomberg, Reuters, FINRA, Intex, FNMA, and GNMA, using their respective APIs and web scraping techniques.
- Implemented robust error handling and retry mechanisms to ensure data reliability.
2. Data Cleaning and Normalization
- Utilized SQL queries to clean and transform the extracted data, handling missing values, outliers, and inconsistencies.
- Implemented normalization techniques to standardize pricing conventions and methodologies across different sources.
3. Data Storage and Processing
- Stored the cleaned and normalized data in a SQL database for efficient querying and analysis.
- Developed Python scripts to process the data, calculate key metrics like price spreads, implied volatilities, and other relevant indicators.
4. Real-time Dashboard
- Built an interactive dashboard using a visualization library like Plotly or Dash to display real-time price information, key metrics, and trends.
- Implemented features to filter and customize the dashboard based on user preferences.
Impact and Benefits:
- Improved Efficiency: Automated data collection and processing reduced manual effort and increased productivity.
- Enhanced Accuracy: Consistent data cleaning and normalization ensured accurate and reliable price information.
- Timely Decision Making: Real-time access to market data enabled timely pricing decisions.
- Data-Driven Insights: The dashboard provided valuable insights into market trends and patterns.
Technical Skills Utilized:
- Programming Languages: Python, SQL
- Data Engineering: Data extraction, cleaning, transformation, and loading (ETL)
- Data Analysis and Visualization: Data analysis, statistical modeling, and data visualization techniques
- Machine Learning: anomaly detection
- Cloud Technologies: AWS
By successfully implementing this project, we significantly enhanced the efficiency and accuracy of pricing US structured finance securities, enabling better decision-making and risk management for the pricing analyst team.