Kaggle House Prices

The competition gives you 1,460 homes from Ames, Iowa with 79 columns each and asks you to predict sale price. It's scored on the error of the log of the price, so being off by 10% on a cheap house counts the same as on an expensive one.

I started by screening features with a few thousand quick pairwise regressions to see what actually moved price, then compared models on a holdout split, both overall and within price tiers. XGBoost and LightGBM were clearly ahead, with Ridge and Lasso behind them.

My first real design was a mixture of experts: a random forest sorted each house into low, medium or high price, and a separate model handled each tier. The router was right 86% of the time, but the errors at the tier edges hurt more than the specialists helped, so I dropped it for a simple weighted blend.

The final notebook adds 26 features I built from looking at the data, things like neighborhood times overall quality, flags for cheap and luxury neighborhoods, and quality by build era, and drops 7 weak ones. Columns where a blank means "none" (no pool, no garage) get their own category instead of being filled in. Four models train on log price with early stopping and get blended at 35% XGBoost, 43% LightGBM, 8% Ridge and 14% ElasticNet.

My holdout score of about 0.102 was too optimistic, because I used the same split for early stopping and for picking blend weights. That's a big part of the gap to 0.121 on the leaderboard. Next time I'd use proper cross validation and refit on the full training set before submitting.