Kaggle Multi Local Module Project Python

Kaggle’s setup is amazing for quick experiments but not great when you start treating your work like an actual project. You get one main notebook. That’s it.

If you want to use your own .py files, you basically have to zip them up, upload them as a dataset, and then import from that path. It’s clunky and hard to maintain. Change one line of code? You need to re-upload the dataset again.

I saw this frustration all over the web, in Kaggle forums, Stack Overflow threads, even Reddit. Everyone was hacking their way around it, trying things like chained kernels or huge notebooks with thousands of lines of code. Nobody seemed happy with it.

So I went down the rabbit hole. Read a bunch of Medium posts, watched YouTube tutorials, skimmed corporate engineering blogs. I noticed a pattern: real ML pipelines in the wild are automated. They have CI/CD. They deploy cleanly. But for personal Kaggle projects, nobody had built something simple and usable.

That’s when it clicked, I could write a small tool that did the boring part for me. A script that could take my local project, package it up neatly, and push it to Kaggle as a dataset and a runnable notebook automatically.

Building kaggle-auto-deploy

The idea was simple:

• Collect all project files.
• Upload them as a Kaggle dataset.
• Auto-generate a notebook that sets everything up and runs main.py.
• Push it, version it, done.

So that’s what I built. A small CLI tool:

python kaggle_deploy.py ./my_project

For example, I used it on a small housing price predictor I’d built.

Behind the scenes, it ties into Git too, so every deployment matches a commit. No more “which version did I upload again?” moments.

Repository: https://github.com/quochung-cyou/kaggle-auto-deploy

Sample: https://www.kaggle.com/datasets/quochungcyou/housing-price-predictor-files https://www.kaggle.com/code/quochungcyou/multi-local-module-project-python-run-sample

Overview

This solution automatically converts any multi-file Python project into a Kaggle-compatible format by:

• Analyzing your project structure and dependencies
• Creating a Kaggle dataset containing all your project files
• Generating a Kaggle notebook that automatically downloads and runs your project
• Uploading everything to Kaggle via API

Guide

You can try command below to try deploy the sample housing price predictor project:

python kaggle_deploy.py ./housing_price_predictor

If you modify the code and redeploy again, you may need use Check Update option to update the dataset and notebook.

Prerequisites

1. Install Kaggle API

pip install kaggle

2. Configure Kaggle Credentials
   • Option A: API Token File
     • Go to https://www.kaggle.com/account
     • Click “Create New API Token”
     • Download kaggle.json
     • Place it in:
       • Linux/Mac: ~/.kaggle/kaggle.json
       • Windows: C:\Users{username}.kaggle\kaggle.json
   • Option B: Environment Variablesexport KAGGLE_USERNAME=”your-username” export KAGGLE_KEY=”your-api-key”
3. Set Permissions (Linux/Mac)

chmod 600 ~/.kaggle/kaggle.json

Installation & Setup

1. Download the Deployer Script

git clone https://github.com/yourusername/kaggle-auto-deploy.git
cd kaggle-auto-deploy

2. Make it Executable (Linux/Mac)

chmod +x kaggle_deployer.py

3. Optional: Add to PATH (Linux/Mac)

# Add to ~/.bashrc or ~/.zshrc
export PATH="$PATH:/path/to/kaggle_deployer"

Usage

Basic Usage

python kaggle_deployer.py /path/to/your/project

Sample Project

This repository includes a sample project called housing_price_predictor that demonstrates how to structure a multi-file Python project for deployment to Kaggle.

python kaggle_deploy.py ./housing_price_predictor

Project Structure

housing_price_predictor/
├── main.py          # Entry point
├── data_loader.py   # Data loading and preprocessing
├── model.py         # Model training and evaluation
├── utils/
│   └── helpers.py   # Utility functions
├── data/
│   └── housing.csv  # Sample data
└── requirements.txt # Dependencies

Running the Sample Project

# Run locally
cd housing_price_predictor
python main.py

# Deploy to Kaggle
python kaggle_deployer.py ./housing_price_predictor