Search Results for: aws

Why I would like to be mentored

12th August 2020 By John McCormack Leave a Comment

Who wouldn’t want to be mentored?

This blog post is created as a direct result of Paul Randal’s offer to mentor people in the SQL Server community. I’ve had a good think about it and it seems obvious to me. Paul asks “Why would I like to be mentored?” I’m going to set out my personal reasons and although this isn’t one of them, my immediate response is “Who wouldn’t want to be mentored?” If you don’t have a big ego and understand that some people know a lot more than you, and have been in the game for longer, it stands to reason that some mentorship could be of a huge personal benefit.

My reasons

I was a late-ish bloomer in tech. I had a different career from ages 16 – 32 so I always felt like I was playing catch up when I started as a DBA. At 32, coming out of university, I landed a job with Monster who just happened to have a large DBA team in Glasgow. I was lucky. I had a senior colleague who was only a month older than me. When I saw how good he was and how much he knew, I felt like I had a lot of catching up to do. But I was also inspired. He mentored me a lot over the 6 years I worked at Monster, as did others, and I benefited from that. I felt I was ready to move on to the next challenge and moved into contracting.

Contracting can pay more, it did for me but it can be a lonely game. You don’t always have the same team spirit that you feel as an employee and any choices about personal development are entirely yours. Sometimes, there is so much dazzling new tech that I don’t know what to learn next. I believe a mentor can help me get focussed. Some of my learning options include:

Focus more on public cloud vendors like AWS and Azure. (Not just relational databases).
Focus more on a specific area of SQL Server that causes people problems and become much better at it.
- Performance tuning
- HA/DR solutions
Improve my PowerShell competence
How to become a better trainer

I am also at a bit of a crossroads in terms of my career. I have the following choices and I don’t know what is best. I hope a mentor can talk me through the pros and cons of each option, and help me honestly evaluate how ready I am for each.

Start consulting (Do I have a niche?)
- Performance consulting
- Cloud migration consulting
- Cloud cost optimisation consulting
Continue contracting
Go back into full time employment but seek a senior position
Launch a service offering a remote DBA service
Develop and launch a remote healthcheck service to identify pain points
Focus on creating engaging online training content
- How to make money from training content but still benefiting the SQL community
- I’m already active in the SQL community, and that will continue whether I’m chosen or not; but I do think that the increased confidence that may come after being mentored by Paul will help me improve the quality and quantity of my content.
- Whether to design and deliver my own content or deliver Microsoft approved courses. (I recently achieved the MCT award but I’m don’t feel ready to deliver MS training yet)

So those are the main reasons that I think an experienced mentor like Paul Randal would be a huge benefit to me and I’d be delighted to be chosen.

Turn the cloud off at bedtime to save 70%

18th January 2021 By John McCormack Leave a Comment

Turn the cloud off at bedtime

When you operate in a public cloud such as Azure or AWS, you pay compute costs for all of the time that your instances are online. For your business facing and business critical systems, that is usually 24/7. But what about all those non production instances that run all night and all weekend? If these don’t need to be online 24/7, you can save up to 70% of your running costs by switching them off. This is what is meant by turn the cloud off at bedtime.

Example SQL Server VM

Your development hours are the standard 9am to 5pm. To allow some flexibility, you wish to make your development instances available to developers from 8am until 6pm.

Weekly hours	Price	Saving %
168	£152.76	–
50	45.46	70.24%

^{* E16s V4 SQL Server VM in UK West, with Dev/Test licensing on Pay as you go pricing model}

Deallocate is important

Ensure that your VM status is set to deallocated in the Azure portal to ensure you don’t receive ongoing billing for your compute expenses. This is best achieved by using the Azure Portal, CLI or PowerShell. If you just shutdown a machine, this won’t deallocate your compute resources and you will still be charged.

Compute savings only

The savings to be made are on compute resources as well as any associated licenses when they are included in your PAYG pricing. These are usually your biggest expenses anyway, but it’s worth noting that you will still be charged for any storage attached to your instance.

PAYG vs Reserved Instances

This post in primarily but not exclusively focussed on PAYG instances. There is a whole post to follow on reserved instances (RI) but I’d like to say this for now: You may still make savings by shutting down a reserved instance out of hours, if you have another instance (perhaps used by a dev team in a different time zone) that can pick up the free allocation. The billing for these is complex however. This page explains how RI reservations work against VMs in more detail.

How to turn the cloud off

When I first did this, it involved a mixture of Automation runbooks, PowerShell and instance tagging. There also wasn’t any flexibility to keep instance on for a bit longer without deleting the tag. Thankfully, Azure have made this process much easier now with this guide on how to start/stop VMs during off-hours.

What can’t be turned off

Certain PaaS services don’t allow the start and stop of services. Azure SQL Database and Managed Instance do not allow the stopping or pausing of the service. This takes this money saving option away from you. You could certainly save out of hours by scaling down your instance on a schedule using automation runbooks. This is a fairly quick operation for SQL DB. As for Managed Instance, it could take up to 4 hours so whilst this isn’t usually a great option for overnights, you could consider it for weekends.

The alternative is dropping and recreating your SQL DBs or managed instances each time using Infrastructure as Code. It’s more work but potentially worth the effort if you are running many and/or large PaaS instances.

What can be turned off

These posts are aimed at SQL professionals so I will stop short of listing everything on Azure however VMs as mentioned extensively in this post are the main choice. They are easy to stop and start manually, and the automation of this process is now easier than ever.

There is a PaaS Service that allows pausing of compute and that is Synapse Analytics. SQL Pools can be brought online for ETL purposes and paused the rest of the time. This can offer significant savings on Synapse Analytics.

Summary

Review all of your non production data services in Azure. If you have a lot, automation is the best and only way to keep fully on top of ensuring you are only paying for the services when you need them. Optimizing costs isn’t a one-time event. You need to be constantly on top of your bill to ensure that new instances are adhering to your cost optimization plans.

As always, please feel free to leave a comment if you wish to join the discussion.

DataGrillen 2022 – Photos and notes

8th June 2022 By John McCormack Leave a Comment

I’m just back from DataGrillen 2022. As ever, I wanted to put together a short post and use the opportunity to share some things I learned, and share some photos of the event. First of all, thanks for such a hospitable welcome from William Durkin and Ben Weissman and congratulations for hosting a very successful event.

On top of everything else that goes into planning a complex event like this, they also had an added responsibility this time to keep people free of covid as far as was reasonably possible. This meant mandatory testing, vaccine passports and masking wearing (inside only). It was a small price to pay after two years of online conferences only.

The barbecue event was on the middle night after the sessions. There was a great choice of food and everything was cooked really well, and of course there were complimentary beers to wash it down.

Following on from the main event which comprised of two days of technical sessions, the host town of Lingen was celebrating the historic Kivelingsfest, a medieval festival dating back hundreds of years. As a result, many of the conference attendees stayed on an extra day to attend the festival which was great fun.

It was home on Sunday but not before we got the chance to celebrate André Kamman’s by taking a boat trip along Amsterdam’s canals.

Datagrillen 2022 photos

Andrew Pruski DeepDive into Docker presentation

DataGrillen 2022 Sessions

These are the sessions I attended. I tried to fit in as much as possible from the two day event.

John Martin – Performing successful cloud migrations – Chalk & talk

It was a very informal delivery by John who guided us through a timeline and the phases of a successful cloud migration. It was a cloud agnostic session so the key information could be used with any major cloud provider.

Andrew Pruski – A deep dive into Docker

I wanted to attend this as I haven’t used Docker a great deal and I wanted to learn more. Andrew provided a great overview and worked through some very well prepared demos. I am keen to try to follow these up by visiting his github repo for the content.

Gianluca Sartori – Time Series for relational people

Gianluca gave a great overview of the differences between relational and time series databases and went on to review a few. He settled on influxdb and demonstrated in detail how it could be used along with Grafana and Telegraf agent (TIG stack) to provide really useful metrics. I do use the TIG stack in my work but I learned about some great settings for dealing with or discarding older data that I plan to take forward.

Monica Rathbun – Performance tuning Azure SQL Database

This session was packed to the rafters and extremely hot, yet Monica’s enthusiastic and engaging style made it so easy to stay focused on the message. The message was delivered well and easy to understand. I took away some key information here regarding query store and readable secondary databases. Overall, it was my favourite session of DataGrillen 2022.

Argenis Fernandez – Ransomware sucks. Beef up your databases against it.

This had to be close contender for my top session. Argenis knew what he was talking about and it showed. He gave very good advice and I have returned home with a checklist that I will need to turn into an action plan.

André Kamman – Looking under the hood of the parquet format

I’m familiar with parquet as I use AWS Athena a lot however I’ve never really looked under the hood. I honestly expected this session to be a bit above me but André’s communication style made it very accessible for attendees of all knowledge levels and easy to follow along.

Jess Pomfret and Sander Stad – Deploying Azure Resources with PowerShell Azure Functions

Jess and I were fellow newcomers at Data Grillen in 2019. I always enjoy her sessions so this was not to be missed. It was my first time seeing Sander and the two presenters worked well to deliver this session. I left convinced that I need to change at least one part of my automation routine to include Azure functions, so I will hopefully be checking them out this week. (If time allows, the trouble with conferences is you return home with SO many ideas)

Heini Ilmarinen – Less Clicking, More Coding! Azure Data Platform Development Using Infrastructure as Code

I’ve dipped my toe into Terraform but I’ve used BICEP and Cloud formation a bit more. After Heini’s session, I get the benefits of Terraform and I’ve added it to my ever increasing list of things to review. A great session in which Heini did well to keep us engaged all the way through.

Robert French – Goldilocks and the three business bears; story telling for business

This was the last session of a very busy two days. They broke the mould when they made Robert and as expected, his session also broke the mould. Robert delivered the session with unmatchable energy, kind of at a blistering pace but the pace was just right at the same time. He got through a huge amount of visual aids. He gave great advice on where you eyes are drawn to in report, how to make key data pop out and how to usual colours and fonts carefully.

Thoughts on sitting DP-300 Azure Database Administrator exam

3rd April 2023 By John McCormack Leave a Comment

On Sunday morning, I sat and passed exam DP-300 which gives the certification of Microsoft Certified: Azure Administrator Associate. I’ve been sitting Microsoft exams for a number of years now but this was my first for a while. I wanted to jot down what thoughts still remained in my head following the exam.

Preparation

I prepared for the exam by following the course from ACloudGuru. I’m a big fan of this site as it has helped me learn so much about AWS; so I thought I would trust it for Azure learning as well. I have to admit I had been going through the course at a snail’s pace and by the time of completion, I had forgotten some of things I had learned initially. I sat their exam simulator and got 77% which was not a pass.

I realised from doing the exam simulator that there were some areas where my knowledge was weaker, specifically on some newer features I hadn’t worked with day to day and also migration options for the likes of Postgres and MariaDB.

I wrote down each wrong answer, learned why it was wrong and what was correct. I resat the test and got 87%. I repeated the process and got 89%. A lot of the questions were repeating by now and I felt ready to sit the exam.

Sitting the DP-300 exam

The exam itself was MUCH harder than the exam simulator, however in fairness, the exam simulator was pushing you to achieve 80% which allowed a bit of a buffer come exam day. (Microsoft only look for 700/1000). There are some areas I think the simulator could have covered more extensively and a bigger bank of questions would have been helpful.

However when it came to the exam, I managed to rely on my experience to pull some old knowledge out of the darkest recesses of my brain, and made a few educated guesses. Thank fully I passed with 800+. Not the best score but the exam itself is pass or fail.

Focus areas for sitting DP-300

I can’t be too specific here due to NDA but I would suggest if you are an experienced SQL DBA but have limited or no recent Azure working experience, some areas to focus on are:

PAAS vs IAAS
1. Remember PAAS needs less administrative effort than IAAS or on premises. Look for clues in the question.
2. Business Critical and Premium vs General Purpose and Standard
3. Azure SQL Server IaaS Agent Extension
4. Database mail for managed instance
Migration options (Online and offline) – And not just for SQL Server
Encryption vs Dynamic data masking
Query store
Key Vault
Some Azure networking like Virtual network and VPN
Know your performance DMVs and how to diagnose blocking and slow executing queries
Backup of system DBs

What’s next

I’m still deciding but it’s given me the bug after a couple of years of not doing much active learning. So I think AZ-104 to help me re-establish some Azure infrastructure knowledge or DP-203 to introduce me to some parts of Data engineering that I’m looking to learn.