AI
The need to study how LLMs work..
When I was 12 years old, I was given a book on transistors, it detailed how they are made of germanium or silicone plates, how it is manufactured. This book of course learned me nothing about electronics and building amplifiers - which was the goal, it just showed that there are some things you really do not need to know, to utilize a technology.
Same goes for AI. Unless you are into development of new AI models and new types of AI, there is no need to know exactly how they work or how to make a new model from scratch. Because you use one of the plenty available.
That you can make use of a chat-AI with no knowledge is easy to realize, but let's look at something more complicated.
Let's say I want to build a chat page for loudspeaker technology, which answers based on my own content and knowledge. It should run local on my computer, it should use my content as knowledge base, it should be trained on my way to express answers and my technology solutions.
There are several stages involved in solving this, and none of those requires in-depth knowledge of how to make AI models. It requires some understanding of software programming and how that works, then more detailed on AI architecture and API calls, and a little about the tools to use.
To start, you can just use on-line chat with a model, provide a prompt where you insert relevant documents. This has some limitations, and it doesn't run locally.
To run a model locally is easy, I can use LMStudio or Ollama. A problem is the computing power of the local machine, it simply can not run the larger models. The models are available, the tools to run them are available.
I can create a page where I use API calls to talk to the model, and now it is easy to switch between using the local model and some off-line provider through API keys.
Then I run into problems with the size of the context, when trying to insert all my documents and content. Then there is something called RAG to solve this, I can scrape all my content and extract keywords that are put into a vector database, which the AI then uses to find information. Maybe it works.
I can train a model on my own content. Use an existing model, train it on my data, then run this model locally on my computer. Now we are getting somewhere.
None of this requires knowledge on how AI models are developed or how they work internally.
In my bookshelf I have a book on neural networks ("AI") from the 90ies I believe. Nice to have, but reading it will not gain you much usable and relevant knowledge on AI as it is today.
I have used AI-powered agents for software dev for some time now. Enough to see patterns in how it works and what it is good at, in its current state.
It is quite good at converting and upgrading existing code. Replace old structure with new updated, upgrade existing code to support newer versions of frameworks around it.
It is quite good at generating new code for things that are generic, well-known. Before making the VideoCall I had no idea what WebRTC is, but no need to know, because the AI agent knows this generic tech and implements it for me. i can then study how it works, if necessary.
It is not good at making new solutions, things that does not exist if you search the web. I need to make out how the solution is supposed to work, then sketch out the architecture. Then I can use the agent to generate code and implement parts, piece-by-piece.
If I do not pay attention, the code it generates is a mess. And it easily corrupts existing codebase.
It does not handle UI and layout well. It struggles with basic implementation, such as placement of UI elements.
All in all, a new era of software development. But the developer is far from obsolete. If you want to create something real, something that is worth creating and useful for others.
This is concerning:
I start watching a video. But the story seems a little off, all images are clearly AI generated, but it's sort of a documentary, well, meant to be, so they are more like just something to support the story, then the narration is also clearly AI generated. But in a documentary the primary value is in the presented facts and what you learn from it, so why not.
The problem is that what is presented is simply wrong. It presents made-up statements and a distorted reality as if they are facts.
A human usually at least try to put together a story based on real-world collected information, something that is reasonably correct. It may be biased due to some agenda that the creator has, but rarely do people just make up the parts that were hard to investigate and find factual information about.
AI has no problem with that, it just makes up something that might be suitable in an alternative reality.
This way, our perception of reality is destroyed, our history is destroyed, and if left unchecked this will lead to a complete loss of knowledge about how the world actually is and how it works.
It is called "programming language" for a reason
When coding, or programming, first started, it was manual by putting machine-code hex numbers into the machines, this then evolved into assembly languages, still just numbers and codes.
The concept of a programming language is words and phrases suitable to describe the functionality of a computer program, which can then be compiled into this machine code by and automated process. A language that can be used to efficiently describe structure and functionality.
With LLM coding agents, we can now skip the programming language, and write in English instead. The idea being that this is easier and more efficient, no need to learn a programming language. But is this true.
A programming language is also a very efficient way to sketch out and describe how something works. Writing software like it is a novel does not really work that well. Automated coding does not remove the need for software developers who understand structure and systems.
Current state of coding agents is that it produces horrible code, if left unchecked and not guided properly. The code is a mess, and while that initially seems like it does not matter, because you don't see the code when you use the application, right? The problem is that the code soon stops working, when trying to add or change functionality. If it worked in the first place.
Automated coding still needs structure and systems. A developer needs to understand and implement this, even if it is embedded into prompts delivered to code automation agents.
A programming language is efficient and provides the structure and rules to describe a software architecture, or system. The coding agent can then be used to increase efficiency by implementing already sketched out structures and systems.
There will always be a requirement to understand how what you are making works, even with no programming language between you and the code. It is just the language describing this functionality that changed. If you are not able to describe and instruct what you are making, it is not possible for any automated tool to create it.
The VideoCall code generation experiment
The VideoCall application is something new from scratch, a perfect opportunity to test code generation from scratch. Let AI code generation make everything, only specify functionality.
I am now only half-way through rewriting the code into something usable.
What was created brings back memories of horror projects from the past, where code is basically a complete mess with everything put into a few functions, there are redundant data objects, it is impossible to make changes without affecting parts in other locations. This is like going bck to the 80ies, where real-time embedded programming was done like this, structured programming, where data is separated from functions, there is no system.
How to solve this? Rewrite the code into a new object-oriented structure, and let the AI move and fix things, as I complete the new structure.
The next application will test a different approach to code generation, where I make the initial structure first, then have code generation make individual parts and pieces.
Regulations and restrictions from govts can potentially create disaster
What if the government in a tech-leading country decides to restrict foreign access to AI models and AI services.
In a small country like Norway, with almost no tech companies, this can have a quite interesting effect. Since there is no native tech, there are no models and service providers in the country, everything needs some sort of service or software from a company in a foreign country.
Oh.
This is what we, as developers need to consider. First, it may mean some tools used in development disappears or gets reduced performance. Today, not necessarily a big deal, depends on what you develop, how you work, what tools you use. Of course if you work on AI, to make new or improved AI services, then restrictions can have an immediate and disastrous effect, effectively stopping your business immediately. Because then you are likely to be reliant on the most recent new models and services available.
So it may have some effect for software development, and potential show-stopper for AI projects.
For everyone else, for generic use, like information search, text generation, picture and video generation, the consequences will not be noticeable, other than you can not use the latest and best models and thus have a meltdown because someone in another country can generate the same annoying slop with a newer model.
If all access is closed, then of course that changes things quite dramatically. Not going to happen.
Rather, this is the disaster - for countries like Norway
The disaster here is perception. How a business and its products and services are seen. Is it top of the line, or is it second tier. This perception will be based on which country the company comes from, and the rating of the country will be based on access to AI tech.
A closed-off Norway will get a very low rating.
Norway is small and will always be dependent on services and products and technology developed abroad, regardless well developed the local tech base is. A small country benefits more from free trade and free flow of goods and services, in fact, its wealth relies entirely on free trade.
Norway does not have any AI providers, so any tech company utilizing AI must source services and technology from foreign providers.
Regulations on use and development of AI to limit what and how
This is the politician's dream, to control the development of tech. The ethical AI, with correct political bias. Can actually happen in Norway, but it will have no significance, since the same politicians have created a tech-toxic business environment and thus there is no AI development of importance in Norway to be affected by their laws and regulations.
Can access restrictions be bypassed?
Maybe, but likely more difficult than simply using a vpn. Is there payment involved, that complicates things, and services generally will be more tricky than accessing a simple web page. For professional use, in a company, there are also legal problems to consider, and if you can not say you used it, then there may be very little value left in it.
One solution is of course to move yourself and the company to a more suitable country. A better solution is to scrap all restrictions and regulations.
AI data centers - some misconceptions (part 1):
Does not bring any high-skill, high-value jobs
It keeps popping up; "New data center in Indre-Granskau brings prosperity and jobs to the community," another data-center is planned.
(While others are cancelled.. )
Data-centers do not create high-skill tech jobs. The AI or the applications or the technology is not developed in data-centers. The data center is just a place where lots of computers and hard drives are located, the location was chosen because it was cheap to build and operate there, perhaps some politician was duped to subsidize with taxpayer money.
The only skills needed for operation is to swap out computer cards and hard drives when they break.
Now, most of you already know this. But it seems like at least a few, still have no clue how computers and software and "the cloud" basically works.
AI data centers - some misconceptions (part 2):
Not the future.
Why this demand for data centers?
Data centers with lots of computing power may be needed now, but will soon become obsolete.
The future of AI is distributed, the computing will be executed on local devices and computers, on local AI models.
Right now we see a surge in data center demand, it happened quite fast, and there are indications the top of demand may already be over. Because we see some data centers are being cancelled.
The panic is caused by service providers of AI tools, media platforms, online services, observing an increase in demand for storage and computing that can not be met by existing capacity.
As is typical for any dynamic system, there is a built-in delay in response, so when turning the dials to compensate, some restraint is usually the better approach, as it very easy to go too far, like when you panic because the temperature goes one degree below comfortable and you turn the heat up to maximum and then when you realize this is getting way too hot, it is too late and you turn off the heat completely and open the windows. A more moderate increase in heating power would work much better to stabilize the situation.
This increased demand for computing and storage is connected to AI.
Now everyone can write long texts, create pictures and even video, using chat-AI. It is easy and fast. You can even automate the whole content creation process, set it up and watch endless videos and posts being uploaded automatically.
But who is going to consume all that content.
Freedom of choice is nice, but the problem here is that each alternative to a video on one subject will be more of the same. And generally, the quality will be very poor. This content has no value at all.
The result is massive amounts of content published only to be consumed by the very same systems that created it; AI makes content that is read and viewed by AI.
Human users are the ones who pay for all this, through advertisement and subscriptions, and this will collapse when user no longer pay for a subscription on something they have no interest in watching and advertisers leave when they see their customers no longer watch the ads.
I wrote AI will evolve into more local processing. Then there is the evolution of LLM AI, which has seen an increase in performance as the models grow larger (more tokens in the model itself and in their context window), there may be low or no gain in further increasing the size, something entirely new is necessary to achieve further significant gains in performance.
China?
China. Again.
Dev companies started to cap AI usage, then moving to cheaper Chinese models because difference in performance makes no significance in practical use.
And those Chinese models are open and can be downloaded and run locally, so you don't pass on all your work and knowledge and ip to whoever host what you used as your coding engine.
Yes, I have used off-site agents and models myself, but be my guest, if you want to steal "code" for what is experimental code snippet made for my personal web site. If I did a project for a customer, could be in defense, likely some part of a real-time control system, I could not do that. Of course I could not.
The business model of charging for code generation is about to collapse, and it is a good thing.
What comes next. For AI.
It doesn't need to be perfect, it can have obvious shortcomings and still be useful. It just needs to give better results compared to not using it.
Current tech is basically an agent that sends a request to an AI model, then gets a reply, then processes and executes.
It is (the agent) tailored for specific use case. Such as coding.
Models for specific purposes
The next step can be models that are also tailored for the same use case. I don't need a model that knows how to do banking in China, when I want to generate code for my simulator application.
This model can then be much smaller and much more efficient, for its intended purpose.
A smaller, more efficient model is easier to run locally, on a computer with limited computing power. I will never have a computer that can run the largest models, but I can set up a local computer here to run something more reasonable, and this smaller model can still perform very well, if it is made/trained for a specific purpose.
Because these models, they do not get smarter simply by having more information in them. They can not utilize information in the same way our brain does, to reason and find answers.
Next step in evolution
LLM seems to have hit a barrier, where further improvement does not materialize simply by scaling up the models and make them larger. There also seems to be limitations with training data, more data does not automatically improve training if the quality of the data is not good.
Expect another future step in evolution, one that dramatically changes training so that the model learns much faster, on less data, more efficiently.
When that happens, a smaller local model can be dynamic, it trains and adapts and learns, as it is being used.