{"id":550,"date":"2026-09-16T12:38:35","date_gmt":"2026-09-16T12:38:35","guid":{"rendered":"https:\/\/codecasm.com\/blogs\/?p=550"},"modified":"2026-09-16T12:43:37","modified_gmt":"2026-09-16T12:43:37","slug":"building-ai-models-from-scratch","status":"publish","type":"post","link":"https:\/\/codecasm.com\/blogs\/building-ai-models-from-scratch\/","title":{"rendered":"Should We Really Stop Building AI Models From Scratch?"},"content":{"rendered":"\n

What an AI event made me question about fine-tuning, originality, and India\u2019s journey toward sovereign AI.<\/strong><\/p>\n\n\n\n

I recently attended an AI-focused event where I got the opportunity to listen to several experts working in the field.
There were around six insightful talks<\/strong>, and what I particularly enjoyed was that the sessions weren’t limited to slides and theory. The speakers demonstrated their ideas through real examples and demos, which made the discussions much more practical.
After the talks, there was a Q&A session.
And that’s where I asked a question give that had been bothering me for some time.<\/p>\n\n\n\n

\u201cI want to build an AI model from scratch. Is that really a bad idea?\u201d<\/h2>\n\n\n\n

I have been experimenting with the idea of building an AI model myself \u2014 not simply consuming an existing model through an API, but actually understanding what it takes to build one.
My intention isn’t necessarily to compete with the world’s biggest AI labs tomorrow.
I simply want to understand the technology deeply.
I want to know:<\/p>\n\n\n\n

    \n
  • What happens when we build a model ourselves?<\/li>\n\n\n\n
  • What does training actually involve?<\/li>\n\n\n\n
  • How does a model learn?<\/li>\n\n\n\n
  • What decisions matter during architecture and training?<\/li>\n\n\n\n
  • Can an individual developer or a small team actually build something meaningful?<\/li>\n<\/ul>\n\n\n\n

    So I asked the experts:<\/p>\n\n\n\n

    \n

    \u201cI am trying to build an AI model from scratch and I want to make it genuinely good. Should I continue?\u201d<\/strong><\/p>\n<\/blockquote>\n\n\n\n

    The response surprised me.
    Almost immediately, the discussion moved toward a different approach:
    Don’t build everything from scratch.<\/strong>
    Instead, take a relatively basic or pretrained foundation model and fine-tune it for your specific use case.<\/strong>
    The reasoning was understandable.
    Training a capable foundation model from scratch requires enormous amounts of:<\/p>\n\n\n\n

      \n
    • High-quality data<\/li>\n\n\n\n
    • Compute<\/li>\n\n\n\n
    • GPU infrastructure<\/li>\n\n\n\n
    • Engineering effort<\/li>\n\n\n\n
    • Experimentation<\/li>\n\n\n\n
    • Time<\/li>\n\n\n\n
    • Money<\/li>\n<\/ul>\n\n\n\n

      And even after investing all of that, there is no guarantee that the resulting model will outperform an existing open model.
      From a practical product-development perspective, the argument makes complete sense.
      But I walked away with another question.<\/p>\n\n\n\n

      Are we solving a problem \u2014 or simply following a trend?<\/h2>\n\n\n\n

      This is where things became interesting for me.
      If everyone starts with an existing model, fine-tunes it, adds a better interface, connects some tools and calls it an AI product…<\/p>\n\n\n\n

      Where does fundamental innovation happen?<\/strong>
      There is absolutely nothing wrong with fine-tuning.
      In fact, fine-tuning is an incredibly powerful technique.
      If my objective is to build an AI system for a particular industry, language, workflow or business problem, starting with an existing foundation model may be the most sensible engineering decision.
      Why spend years rebuilding something that already exists?
      But there is another perspective.<\/p>\n\n\n\n

      What happens if nobody tries to build from the ground up?<\/h3>\n\n\n\n

      Imagine if every developer followed the same logic:<\/p>\n\n\n\n

      \n

      \u201cSomeone has already built the foundation. Let’s just use it.\u201d<\/p>\n<\/blockquote>\n\n\n\n

      We would certainly get better applications.
      We would get specialized models.
      We would get AI products.
      But would we also develop the deeper capabilities required to build foundational AI ourselves?
      That’s the question I couldn’t stop thinking about.<\/p>\n\n\n\n

      The Indian AI question<\/h2>\n\n\n\n

      This becomes even more interesting when we talk about sovereign AI<\/strong>.
      India is talking increasingly about developing its own AI capabilities \u2014 models that understand Indian languages, Indian contexts and Indian requirements, while reducing excessive dependence on technology controlled elsewhere.
      But what exactly do we mean by an Indian AI model<\/strong>?
      Is it:<\/p>\n\n\n\n

        \n
      1. An Indian company fine-tuning an existing foreign foundation model?<\/li>\n\n\n\n
      2. An open-source model trained further on Indian datasets?<\/li>\n\n\n\n
      3. A model whose training data, infrastructure and weights are controlled in India?<\/li>\n\n\n\n
      4. Or a genuinely foundational model trained from the ground up in India?<\/li>\n<\/ol>\n\n\n\n

        These are not necessarily the same thing.
        And perhaps they shouldn’t be treated as the same thing.
        A fine-tuned model can absolutely be valuable and can serve millions of Indian users.
        But if our long-term objective is technological independence, shouldn’t there also be people experimenting with the fundamentals?<\/p>\n\n\n\n

        \u201cIt’s a waste of time.\u201d<\/h2>\n\n\n\n

        One phrase from the discussion particularly stayed with me.
        The idea that building a model from scratch could be a waste of time<\/strong> because it is expensive, difficult and time-consuming.
        Technically, I understand the argument.
        If your objective is to launch a product quickly, it probably doesn’t make sense to spend years building infrastructure that you could otherwise leverage.
        But I don’t think \u201cnot commercially efficient\u201d<\/strong> automatically means \u201ca waste of time.\u201d<\/strong>
        There is a difference between building something from scratch to launch a startup next month and building something from scratch to learn, experiment and develop capability<\/strong>.
        The second objective can be completely different.<\/p>\n\n\n\n

        We didn’t learn programming by only using libraries<\/h2>\n\n\n\n

        Think about software development.
        We use frameworks.
        We use libraries.
        We use operating systems.
        We use databases.
        We use APIs.
        Nobody expects every developer to build a database engine before creating an application.
        And that’s perfectly reasonable.
        But there are still people who build databases, operating systems, compilers, programming languages and networking protocols.
        Why?
        Because someone has to understand the layer underneath.<\/strong>
        AI shouldn’t be any different.
        We can use existing models for applications while simultaneously having researchers and engineers working on foundational models.
        Both ecosystems are necessary.<\/p>\n\n\n\n

        Maybe the answer isn’t \u201cfrom scratch OR fine-tuning\u201d<\/h2>\n\n\n\n

        Perhaps I was asking the wrong question.
        Maybe this shouldn’t be a battle between:
        Build from scratch<\/strong>
        vs.
        Fine-tune an existing model<\/strong>
        Instead, we should think of it as a technology stack.<\/p>\n\n\n\n

        Layer 1 \u2014 Use<\/h3>\n\n\n\n

        Use existing AI models to solve real problems.<\/p>\n\n\n\n

        Layer 2 \u2014 Customize<\/h3>\n\n\n\n

        Fine-tune and adapt models for specific domains, languages and workflows.<\/p>\n\n\n\n

        Layer 3 \u2014 Research<\/h3>\n\n\n\n

        Experiment with architectures, training techniques, datasets and optimization.<\/p>\n\n\n\n

        Layer 4 \u2014 Build<\/h3>\n\n\n\n

        Train increasingly capable models from the ground up.
        A healthy AI ecosystem needs all four.
        Not every developer needs to work at Layer 4.
        But if an entire country only operates at Layers 1 and 2, it may become dependent on the people controlling Layers 3 and 4.
        And that is the part I find worth discussing.<\/p>\n\n\n\n

        My takeaway from the event<\/h2>\n\n\n\n

        I went to the event expecting to learn about AI.
        I came back thinking about something much bigger:<\/p>\n\n\n\n

        What does it actually mean to build?<\/strong><\/p>\n\n\n\n

        There is tremendous value in standing on the shoulders of giants.
        We shouldn’t reinvent everything simply because we can.
        At the same time, we shouldn’t become so comfortable consuming technology that we stop experimenting with its foundations.
        Fine-tuning isn’t the enemy.
        Using existing models isn’t the enemy.
        APIs aren’t the enemy.
        The real question is whether we are developing enough deep technical capability<\/strong> alongside building products.
        Because someday, if we want to say:<\/p>\n\n\n\n

        \n

        \u201cThis is an Indian foundational AI model.\u201d<\/strong><\/p>\n<\/blockquote>\n\n\n\n

        someone has to be willing to start much earlier \u2014 when the model is small, the experiments fail, the GPUs are limited, the dataset isn’t perfect and everyone around them says:<\/p>\n\n\n\n

        \n

        \u201cWhy are you doing this? Just fine-tune an existing model.\u201d<\/strong><\/p>\n<\/blockquote>\n\n\n\n

        Maybe they will fail.
        Maybe the model will never become competitive.
        Maybe the project will take years.
        But even then, something valuable might be created:
        knowledge.<\/strong>
        And knowledge compounds.<\/p>\n\n\n\n

        Conclusion<\/h2>\n\n\n\n

        My conclusion after the event isn’t that experts are wrong.
        They have very valid reasons for recommending fine-tuning. For most real-world products, starting from an existing model can be dramatically more practical, economical and faster.
        But I also don’t think we should discourage experimentation simply because something is difficult.
        If everyone only optimizes for what is commercially efficient today, we may build excellent products without necessarily building the foundational capabilities required for tomorrow.
        Use existing models. Fine-tune them. Build products.<\/strong>
        But somewhere, someone should also be asking:<\/p>\n\n\n\n

        \n

        \u201cWhat if we build the model ourselves?\u201d<\/strong><\/p>\n<\/blockquote>\n\n\n\n

        Because perhaps the journey toward sovereign AI doesn’t begin when we have a massive model.
        Perhaps it begins when someone decides to understand how to build one.
        And perhaps my little experiment of building an AI model from scratch isn’t a waste of time after all.
        Maybe it’s simply a different kind of investment \u2014 an investment in learning what lies beneath the AI we use every day.<\/strong><\/p>\n\n\n\n

        \n\n\n\n

        What do you think?<\/h3>\n\n\n\n

        Is building an AI model from scratch unnecessary when powerful open models already exist?
        Or should developers, researchers and countries like India continue experimenting with foundational models \u2014 even when it isn’t the fastest or cheapest path?
        I’d genuinely like to hear different perspectives.<\/p>\n","protected":false},"excerpt":{"rendered":"

        What an AI event made me question about fine-tuning, originality, and India\u2019s journey toward sovereign AI. I recently attended an AI-focused event where I got the opportunity to listen to several experts working in the field.There were around six insightful talks, and what I particularly enjoyed was that the sessions weren’t limited to slides and […]<\/p>\n","protected":false},"author":1,"featured_media":551,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[22,23,14,10],"tags":[24,7,36,12],"class_list":["post-550","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","category-codecasm","category-free","category-ngpune","tag-blog","tag-codecasm","tag-community","tag-ngpune"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/codecasm.com\/blogs\/wp-content\/uploads\/2026\/09\/scratch-AI.png?fit=1536%2C1024&ssl=1","_links":{"self":[{"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/posts\/550","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/comments?post=550"}],"version-history":[{"count":2,"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/posts\/550\/revisions"}],"predecessor-version":[{"id":554,"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/posts\/550\/revisions\/554"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/media\/551"}],"wp:attachment":[{"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/media?parent=550"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/categories?post=550"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codecasm.com\/blogs\/wp-json\/wp\/v2\/tags?post=550"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}