status-updates

Week 4 Status Updates

Monday: Refactoring Adventures

Python Package Development Code Refactoring 🏃‍♂️

Morning Session 🌞

Started the day with a focus on refactoring the code base of sensitive_data_detector package based on the feedback from Amit Sir, making it more efficient and readable learnt how to introduce OOPS concept in the code.

oops.jpg

Technical Improvements 💪

pr_mg.jpg

Pull Request & GitHub Issues 📚

Hovering over Licenses 📝

Learnings 🎯

Today I learnt about:

A day of overcomming challenges and refactoring! 📈

Tuesday: Open Source & Learning Journey 📚

Morning Inspiration 🌟

Mentorship Session 💭

Had an insightful session with Amit sir and Div sir where:

Technical Deep Dive 🛠️

Pandas Local Setup

Open Source Contribution Prep

Blog Writing ✍️

Wrote a detailed guide on setting up Pandas locally:

Git Essentials 🌿

Learned crucial git concepts:

A day full of learning and preparation for open source contributions! 🚀

Wednesday: Grind & Shine 😎

Networking & API Development Journey 🌐

Computer Networking Fundamentals 📡

networking.png

Studied networking basics from GeeksForGeeks, covering:

API Deep Dive 🔄

Explored different types of APIs and their use cases:

api.png

1. REST API

2. GraphQL API

3. SOAP API

4. WebSocket API

HTTP Methods Study 🔀

Learned about core HTTP methods:

AWS Infrastructure Components ☁️

aws.png

Studied AWS networking basics:

FastAPI Implementation 🚀

fastapi.png

learned about fastapi and made my own test server , fast api is a modern, fast (high-performance), web framework for building APIs it uses starlette for the web parts and pydantic for data handling

fastapi.png

Built my first FastAPI test server:

Example of my test server code:

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

@app.get("/items/{item_id}")
def read_item(item_id: int):
    return {"item_id": item_id}

Best Practices Learned from Stack Overflow 🎯

best_practices.png

  1. API Naming Conventions
    • Use nouns instead of verbs in endpoints
    • Use plural nouns for consistency
    • Examples:
      • ✅ Good: /users, /items, /orders
      • ❌ Bad: /getUser, /createItem, /deleteOrder
  2. Resource Hierarchy
    • Keep URLs clean and logical
    • Use proper nesting for related resources
    • Example: /users/{id}/orders
  3. HTTP Methods Usage
    • GET for reading
    • POST for creating
    • PUT/PATCH for updating
    • DELETE for removing
  4. Status Codes
    • 200: Success
    • 201: Created
    • 400: Bad Request
    • 404: Not Found
    • 500: Server Error

    status_codes.png

  5. Query Parameters
    • Use for filtering, sorting, pagination
    • Keep names clear and consistent
    • Example: /items?sort=desc&limit=10

A productive day of learning modern web technologies and implementing them! 💻

Thursday: Cloud Deployment & Real-time Communication 🚀

AWS EC2 Deployment 💻

  1. Created and Set Up EC2 Instance
    • Launched new EC2 instance
    • Connected locally using SSH
    • Installed required packages:
      • FastAPI
      • Uvicorn server
  2. Server Deployment
    • Created test FastAPI server
    • Deployed on EC2 instance
    • Accessed using public IPv4 address
    • Tested on port 8000

aws-server.png

FastAPI & Pydantic Implementation ⚡

pydantic.png

Real-time Communication Deep Dive 🌐

real-time.png

Protocol Study

Voice Agents & Real-time APIs

voice-agents.png

Communication Types

voice-agents2.png

LiveKit Exploration

livekit.png

WebRTC Framework

webrtc.png

Voice Agent Pipeline

User Audio → Speech-to-Text → LLM Token → Text-to-Speech → User Audio

pipeline.png

Key Learnings 🎯

A day full of practical implementations and learning about real-time communication! 🚀

Friday: Voice Agent Development with VAPI.AI 🎙️

Project Overview

Created a voice agent system using VAPI.AI after discussing requirements with Anant. The project focuses on making automated calls and handling call data efficiently.

locally.png

Technical Implementation 🛠️

1. VAPI Integration Setup

2. Phone System Configuration 📞

3. Server Implementation 🌐

4. Public Access Setup 🔄

Data Flow Architecture

POST Request → VAPI API → Twilio Call → Server Events → Local CSV

Key Learnings 🎯

Successfully implemented a voice agent system with real-time call handling and data logging! 🚀

Check out the complete project here: Voice-Agents Repository

Saturday: PySpark & Databricks Deep Dive 🚀

PySpark Architecture Study 🏗️

PySpark Optimization Techniques 💡

  1. Data Optimization
    • Partition tuning
    • Caching strategies
    • Memory management
  2. Query Optimization
    • Using filter and where clause
    • Join optimizations
    • Broadcast joins
    • Minimizing usage of collect and shuffle operations

Directed Acyclic Graphs (DAGs) 📊

Databricks ML Pipeline Implementation 🔄

Created a mock ml pipeline in databricks and ran it

Pipeline Components

  1. Data Ingestion
    • Raw data loading
    • Initial preprocessing
    • Schema validation
  2. Data Processing
    • Feature engineering
    • Data transformation
    • Cleaning operations
  3. Model Training
    • Algorithm selection
    • Hyperparameter tuning
    • Cross-validation
  4. Model Evaluation
    • Performance metrics
    • Validation checks
    • Error analysis

workflow.png

workflow2.png

Exploring Mlops

mlops.png

mlops2.png

Key Learnings 📚

A productive day of learning advanced PySpark concepts and implementing ML pipelines! 🚀

This site is open source. Improve this page.