Year of waterfall, performance improvements

I spent majority of my free time this year working on a complete redesign of the current: http://tiltbook.com. I will skip the product side (features added) and will focus on the technical one.

Basically the whole frontend facade was redesigned, the templating was moved from backend (.twig + jQuery) to React while communicating, once again, with fully replaced backend API.

Why?

FE? => user experience.

BE? => my legacy code sucked horribly, adding new features was nearly impossible and as the components rendering was moved to fully dynamic FE the change was simply a must.

Visual comparison:

Before.

tb_visual_before

After.

tb_visual_after

 

Funny facts about the size of the change from codebase perspective:

Number of commits my feature branch is ahead from master + number of files changed.

Screenshot from 2016-12-18 17:01:51 Screenshot from 2016-12-18 16:59:02

Performance improvements

I knew at the beginning what is the performance bottleneck. Long, long, loooong php methods holding many different business rules trying to render the appropriate statuses for user’s newsfeed.

So… after a year of rewriting the code, feature after feature, query after query, cleaning codebase and asking myself how the fu** this code works and WHY is so ugly while wishing for someone to jump in and release me from this never ending one year long suffering I realised I am finally adding the last last touches and is time for some performance optimization!

The biggest performance as well codebase optimization happened when I moved all the business logic related rendering of statuses from PHP to one advanced more complicated MySQL query which I later split to two parts in order to improve the performance even more. One part responsible for retrieving a list of xy statuses IDs and to a part that will fetch all FE required data to display them.

Legacy, 1582 lines long class responsible mainly for fetching right statuses:

Screenshot from 2016-12-18 16:10:07

Replaced (more or less) with some handy queries like:

SELECT 
    os.status_id
    COUNT(DISTINCT sl.id) - (SUM(sl.type) / COUNT(DISTINCT us.status_id)) AS dislikes_count
    ...
FROM   
    user_status us 
INNER JOIN user_status os 
    ON us.original_id = os.status_id 
INNER JOIN status s 
    ON os.status_id = s.status_id 
LEFT JOIN article a 
    ON os.status_id = a.status_id 
INNER JOIN tiltbook_users u 
    ON us.user_id = u.id 
INNER JOIN tiltbook_users ou 
    ON os.user_id = ou.id 
LEFT JOIN user2group u2g 
    ON os.destination_user_id = u2g.user_id 
LEFT JOIN groups g 
    ON g.id = u2g.group_id 
LEFT JOIN user2challenge u2c 
    ON os.destination_user_id = u2c.user_id 
LEFT JOIN challanges c 
    ON c.id = u2c.challenge_id 
LEFT JOIN status_like sl 
    ON os.status_id = sl.status_id 
WHERE us.status_id IN (a list of xy status IDs)
...

Current stack:

  • Apache
  • PHP 5.4
  • MySQL 5.4
  • Symfony 2.3

New stack:

  • Nginx
  • PHP 7.0
  • MySQL 5.7 (+ completely reworked status related tables)
  • Symfony 3.2

As u can see the technologies were drastically updated and the code base was fully rewritten but did it bring any performance boost?

I used blackfire for profiling and here is the compared result:

Page to render:

old_home_vs_new

Ajax call to load statuses:

old_get_status_vs_new

Current stats:

At the end I decided to remove the AJAX request responsible for loading statuses in the user’s newsfeed and fetch them directly in the first initial page load to provide even better user experience.

Here are the performance stats for the newsfeed page ALREADY containing the statuses.

final_stats

Pretty impressive I have to say when u combine the charm of PHP 7 with some optimized queries.

wow

impressed

Pro tip

Careful with AJAX calls and symfony. I added a sleep(5) at the beginning of these two methods and the following happened:

sleep

This is happening due to session locking. Solution? In case you are not going to modify the session in your AJAX call, make sure to free the lock from session file by:

Screenshot from 2016-12-18 18:00:30

Future

Of course just updating the technology stack is not enough and many little things had to be done and also some time spent with the server configs but in order to keep the blog post short I didn’t mention them.

There are definitely still many things that could be improved in the codebase as well from the performance perspective but luckily the project release is next week so I don’t have time for them and hopefully in a year I will realise how many things I overlooked or how many SOLID principles I accidently broke.

Now…. Christmas, starcraft2 and “chill” ;)