“Instead, we turn to the other strategy, which is the one Birbier used: make the problem harder. That doesn’t sound promising. But when it works, it works like a charm.”
[How not to be wrong, Jordan Ellenberg]
When your friend was struggling with a difficult problem, we often said: “Don’t make it complex, just start with a simple problem”. This is because we have experienced that this simplification provides some clues for solving the difficult problem. This is what mathematicians actually do every day. When proving some statements, they start from the simplest case and expand it to the target problem. However, sometimes, making the problem harder suggests a simple alternative way to solve your real problems effectively.
Many data scientists have focused only on reducing the number of features to make a data-driven model simper. However, this approach does not always give the simplest model. The projection onto the low-dimension (fewer features) may make the data structure more complicated, leading to a failure of spotting the hidden pattern. Hence, sometimes, they need to increase features to make a model simpler (because of more data, more simple). This alternative thinking (adding more features) embodies the trade-off between a simpler model with many features and a complicated model with few features.
Everybody makes DATA 