A Bug, A Brute Force Fix, and a Voila Moment
Hi Everyone,
I want to take a moment to share my learning from a rather prolonged fight with a bug and the eventual bug fix. It was a test of resilience and determination and I feel its worth talking about.
What is the bug?
A rather simple problem in our FO Angular application where dates were not being formatted correctly across all screens.
Sounds straightforward, right? But this wasn't just any bug; it was first noticed on March 23, 2025, and by the time I got it, (April 21), it had already consumed a significant amount of time and effort from other devs. They'd meticulously analyzed the root cause, only to find that everything was absolutely normal! – no changes in code, configuration, data, or environment. It was a perplexing mystery, and I was stepping into a bug that had already made others feel pretty stumped.
How did that tough bug finally succumb to a fix?
I was informed about the approaches that were already tried and ensured that the issue isn't from the back-end. This helped me narrow down the area of focus entirely to the front-end. Now, one crucial piece of information is that the same code is working absolutely fine in local but not in a deployed environment link dev, stage, prod. I started with the question, what is the difference between local environment and deployed environments? I went into the jenkins file and looked at various stages and env variables. One lead I got from it was that it freshly installs every package and freshly build it every time. In local we don't usually re-install packages from scratch when we switch branches/update code. So I freshly re-installed all the packages and bug started showing up in local as well. This ensured that the issue is not with the code, it's with the package installations.
Within our front-end application, we are using a library called ‘moment’ which we use to format dates throughout the application. Since the issue was with the date format, I naturally started looking into the moment library thinking that's where the issue is.
I used all the resources that were at my disposal - AI tools, Github copilot, help from other developers, information about moment library on the internet, everything pointed me towards one direction - the issue is with the moment library. I tried 15 different approaches, raised 6 PRs, and at least 10 deployments within 4 days. Every time I raised a PR, it was working fine in local but used to break again as soon as I deployed it in Dev.
I felt frustrated, clueless, lost, and resigned (I was actually losing my mind at this point 😅)
Finally on Friday, I decided to do it the hard way of debugging - using brute force debugging approaches. I had to manually debug 26,000 lines of code in package-lock.json to find out which is the exact library causing the bug. I did nearly 20 iterations deleting and reinstalling everything again and again, comparing the whole file with the main branch every single time and narrowing down the scope of the bug with every iteration.
Finally, I found out html-entities is the library which was causing this bug. This library isn't a directly installed package but an internal dependency of another package we installed in our application. This is in no way related to the moment library, dates, date formatting, or even the grids we use. It was a totally unanticipated, uninformed, and unknown library. I went to into the library and looked into its commit history, and realised they had commits on and recently before 23rd Mar (when the bug was noticed for the first time) This reassured me that I'm looking at the right library. I made the changes, pushed the code and the date format started showing up in Dev ( 🥁🎉🥳 ) This was the "Voila!" moment for me. I slept peacefully that night 😅
Why am I sharing this here?
As developers, our job is the find the fastest and most optimal solutions for our project, this include quick ways of solving problems like using AI tools, searching on the internet, taking help from peers, e.t.c which works most of the times.
But at times like this when all of the above approaches fail, we usually give up and feel resigned about it. That is when brute force becomes the most likely approach that can solve the bug. If I hadn't gone through all the 26,000 lines manually, even if the entire team sat on this issue for weeks, I don't think we would have gotten anywhere close. It was so frustrating and exhausting for sure, I even had to stay up till 10PM on a Friday night to make sure it's working, but the satisfaction we get solving bigger problems is worth all of it. I am going to remember this bug for a very long time and every time I recall this incident, I will feel happy and proud about this.
I have some takeaways from this experience:
- Local ≠ Deployed: Dependency trap is a real
Lesson: Always test with a clean node_modules (e.g., rm -rf node_modules && npm install). Local caching masks issues; Jenkins-like fresh installs expose them
- # When AI/Peers/Google fail, brute force debugging is a decent option
- Timestamps in Commit Histories Are Clues
Lesson: Check dependency changelogs/commits around the bug’s first sighting. A library’s "unrelated" update might break your app
- The ‘Works on My Machine’ Trap is a Silent Killer"
Lesson: Standardize dev environments (e.g., Docker, nvm, CI-mirrored setups). If a bug only appears in deployment, it’s likely an env mismatch—not your logic.
Thanks a ton for reading this. Have a fantastic day ahead!
.webp)
When to Hire CodeWalnut?
