Categories
General

Update

Last month I started working as an independent software developer. I was able to find 2 contracts writing .NET code. I was able to find the first job thru a referral. The other job was listed in linkedin. Both of these jobs allow me to do my work remotely. In this post I want to provide a quick update on these 2 contracts.

CRM Integration

In this job, I’m integrating 2 CRM providers ConnectWise and AutoTask. These CRM providers allow you to create companies, contacts, tickets, opportunities, etc. I’m using Visual Studio Team Services as our source code management tool. We’re using web forms for this project. On the JavaScript side, we’re using jQuery, Knockout, Bootstrap.

SMS marketing platform

In this job, we’re creating a new sms marketing platform. We’re using Twillio, Docker, AWS, .NET Core 2, ServiceStack, and Git. One of the challenge I have faced in this job is to learn these technologies since my knowledge was limited in these areas. Let me give you an example. Every time we commit code, our code base is built using Travis. After code is packaged, it gets deployed to Docker. Since I’m new to Docker, I had no idea how to debug code in a Docker container. The strange thing was that my code worked locally but the same code was not working on QA. However, after asking other developers, we came to the conclusion that the issue was not the code. It was an issue with our deployment. During a git merge, a line of code was removed that affected our deployment logic.

That’s it for now. See you next month.

Categories
General

Using AWS Python SDK to Read Simple Queue Service Messages

In this post, I want to share how to use AWS Simple Queue Service to manage email bounces and complaints. In my previous post, I wrote about sending emails using AWS Python SDK. First, we have to setup 2 SNS topics to handle email bounces and complaints. Go to the Simple Notification Service console and add a topic. After creating a SNS topic, you should see something similar to:

Pay attention to the protocol column above. The protocol for my email-bounce is set to SQS. This is very important since we want to process SQS messages every time SES receives a bounce or complaint. The endpoint is set to a SQS queue.

Now, let’s go back to the SES console and add a configuration set. Here is my SNS destination named email-bounces which is link to the email-bounce topic we created before.

During the sns setup, we also have to create a sqs queue to hold our messages.

Since SNS will be responsible to act as a proxy, we have to setup permissions for this queue to receive SNS messages.

After completing these steps, we are ready to read messages using Python.

I created a very simple Python script. At the top, I’m importing boto3 and my constructor is creating a sqsClient. To read messages, I created a method that takes an url as an argument. In line 24, you can see that the queueUrl is the location of our sqs email bounce. To actually read the message, we call receive_message and pass the sqs url and also MaxNumberOfMessages. In this case, I’m just using 1 to keep things simple. And finally we return the message.

I also have another method called GetEmailAddress that takes in a message as an argument. First, I retrieve the body string and start parsing it to find the email address. At the end of this method, I return the email address.

With this email address, I can clean up my records so I don’t send emails to it. That’s it for now. Next week, I’ll continue using Python scripts to interact with DynamoDB.

Categories
General

Sending emails with AWS Python SDK


Houston we have a problem. Not Houston. I’m the one with the problem. I need to send emails to my list of users but I know that some of these users were spam users. I need a way to send emails and also remove invalid emails from my system.

In the past I have used AWS .NET SDK to send emails and also track bounces and complaints. But I want to challenge myself so I want to play with the Python SDK. In this post, I want to share how to send emails using the AWS Python SDK.

Install the python SDK by following these instructions. Make sure to setup a profile that has permissions to send emails. In this example, I’m using a profile name “python-scripts” that has permissions to send ses emails. Make sure you set this name based on your setup.

After installing boto3, we are ready to create a class responsible to send emails. Take a look at EmailController.py below:


Now let’s create a class that uses EmailController.py. Take a look at EmailRunner.py.

I like to keep things simple. I have a csv file with all the email addresses. Each line in the csv file has an email address. Next, my class reads each line and sends an email. If there are any errors while sending the emails, I’ll use another csv file to keep track of these errors.

If we were able to send emails successfully, we will receive a valid response with status of 200.

That’s all for today. In a future post, I’m going to show you how to setup sns notifications for bounces and complaints. Have a nice day!

Categories
.Net AWS CodeDeploy General

Creating AWS CodeDeploy Deployments Using .NET SDK

This is part 3 in a series dedicated to AWS CodeDeploy API using .NET SDK. In part 1, I created the CodeDeploy application using ASP.NET Core MVC, C#, and DynamoDB. In part 2, I created the deployment group which has settings for alarms, load balance, auto scaling groups, deployment styles, and other settings.

In this post, I want to concentrate on creating the deployment. In the deployment request, you can specify revision information (Github or S3 settings), deployment group, auto rollback configuration, and other settings.

Talk is cheap. Show me the code!

If you want to follow along, you can visit my github repo at https://github.com/agileraymond/DotNetDeployments. Now that you have a reference to the repo, let’s modify our controller so we can display our view.

This controller action displays our AddDeployment view.

There is a limitation with this view since it only displays S3 settings. In a future post, I will revisit this view and add the Github repo as well. I wanted to get something working in a short amount of time. When the user clicks on Add Deployment button, it will call a Post controller action to trigger a new deployment.

If the user entered all required information, a new deployment will be created in the AWS CodeDeploy console.

If you want to read more about the AWS .NET SDK, follow this link https://docs.aws.amazon.com/sdkfornet/v3/apidocs/Index.html.

If you have any questions or issues with this code, contact me via twitter @agileraymond.

Have a nice day!

Categories
General

How to be Consistent with your Writing

I started writing on this blog on April 2014. This was my second attempt in starting a blog. During my first attempt back in 2013, I was very excited to start writing and sharing with my fellow software developers. I bought a domain and paid for hosting thru GoDaddy. I had big dreams with this new project but I could not write. I didn’t produce a single article. When it came time to renew my domain, I decided to cancel my hosting account and also not to renew my domain. I came up with so many excuses not to write anymore.

However, with the start of the new year, I decided to create a new blog. This time I decided to use solutionsByRaymond.com because I wanted to share solutions to common software problems. My goal was to create 1 article for month. So far I have only missed couple of months. To stay consistent on my blog, I read everyday. Currently I’m concentrating on cloud computing articles. AWS is my favorite cloud provider and there are blog posts everyday. My second favorite topic is .NET framework, C#, and its related tools. Let’s not forget about HTML, CSS, and JavaScript.

After reading, I like to write for a few minutes using my cell phone. I use Evernote and take notes without paying attention to grammar or errors. During this time, I want to capture my ideas because I will forget later. When I’m using my MacBook Pro, I write directly on my blog. I also use Visual Studio Code to take notes. When I’m ready to publish an article, I read it multiple times to find any errors.

This is my advice to other developers starting with a blog. Make it a habit to write everyday. It doesn’t have to be 500 or 1000 words. Your goal should be to write something everyday. This will help you stay consistent with your writing. If you write something this week and don’t produce any content the following 2 weeks, it’s going to be difficult to stay consistent.

Stop reading now and go write something. See you next time.

 

 

Categories
General

Solutions Not Problems

If you listen to the news, most of the time it’s bad news. If you listen to a co-worker, they are always complaining about other co-workers. Sometimes they complain about their own jobs. They complain about their boss. They complain about rules and regulations. We seem to be inundated with problems and complains. Before I started writing on this blog, I took days to come up with a good domain name. I use raymondsanchez.com and others domain names but they were not perfect for the content I wanted to share.

In the software world, we need business owners, business analyst, software developers, quality assurance testers, and other personnel to create solutions, mobile apps, software products. In my role as a software developer, I need to build solutions not problems. I need to contribute high quality software. I need to fix bugs in a timely manner. My point is that I have to concentrate more on the solutions than on problems.

That’s why I decided to purchase SolutionsByRaymond.com where I’ve shared articles on software development. Most of my articles are related to the .NET ecosystem and tools that have been useful in my 16 year career as a software developer.

Remember this: concentrate your efforts in solutions and not your problems.

Categories
.Net Agile General

Last Deployment of 2017


Yesterday we completed our last deployment of 2017. Web sites, WCF services, Web Api, Windows Services, and a PowerBuilder app were deployed without major issues. Our sprints run for 2 weeks and we tried to deploy at least once a month.

In this post, I want to share some of the tools we use at MDBuyline to ship high quality software.

Atlassian Jira
Our business analysts meet with our internal users on a regular basis. Sometimes these conversations take place informally during lunch or when they meet at the hall. The conversation goes something like this: “hey Bob, how are you? By the way, I wanted to show you an issue that I’m having with our website.” During these conversations, our analysts will gather enough information to create a user story. Bob will login to Jira and provide a brief description of the issue or feature request. The more information they provide the better. Adding visual aids also helps developers understand these stories better.

These stories will go to a backlog. Every other week we (developers and BA) will meet to go over these stories and also to set priorities for our next sprint. Currently we are in Sprint 47 and it will end next Thursday. Our sprints are 2 weeks long. We also have a scrum master that helps the team manage stories, priorities and deadlines.

Continuous Integration

We currently use subversion for our source control but will be migrating to git soon. Before we start working on a new story, we create a new branch based on the latest change of trunk (master in git). To continuous build our projects, we use Jenkins to handle our .NET builds and also our deployments. All of our projects have a Jenkins job that is set to build against the trunk version. The hassle with working with branches is that we have to ask our team lead to add a new Jenkins job and point it to our new branch. Hopefully with out Git migration, these setup will be handle automatically.

Continuous Deployment

After Jenkins completes building our projects, we use different PowerShell scripts to deploy web sites and web services to the right environment. We have 3 environments: dev, test, and prod. Let me give you an example so this becomes clear. Let’s say that we have a .NET website and it needs to be setup with Jenkins. Our team lead will add a new Jenkins job and set it up with subversion authentication, msbuild arguments, and PowerShell scripts. This job is build against the trunk version. Since we have to create a branch, our team lead will create a new Jenkins job based on the job that points to trunk. After creating the new job, the team lead will modify the url to point to the branch instead of the trunk url. At this point, we should be able to trigger a build and create artifacts. To move these artifacts from our Jenkins server to the correct environment, we use PowerShell scripts to copy files from Jenkins to server dev-win for example.

Summary

This year is coming to an end. For me personally, it has been a great year adding new features to our software products. Our production environment is in great shape since we ship high quality code. Jira helps us create sprints with the right items to work on. Jenkins helps us with our continuous integration. PowerShell scripts deploy our software efficiently and in a timely manner.

 

Categories
General

How to be an Effective Support Engineer

effective support engineer

For the last 2 months I’ve been in charge of our production support. If there is a bug or something needs investigation, a ticket is created to start this work. We have websites, web services and windows services to manage all these products. We are a .NET shop but also run linux servers. To be an effective support engineer, you have to familiar with your codebase. Some will say that you have to be an expert. The more familiar you are with your code, the easier it becomes to resolve issues.

How to become a project expert?

It is going to be very difficult to become an expert without having spent enough time on a given project. One way to become a project expert is to be familiar with the code. You have to be familiar with the architecture and how the project is structured. First step is to add new features to the project. By adding new features, you will understand the project better than if you start by working on bugs. Pair programming is another way to gain project knowledge. Ask other developers for help. During this knowledge transfer session, take notes and share it with other developers. The next time you receive a bug or ticket on this project, you will be in a better position to resolve it in a timely manner.

Take notes

On many occasions, I receive the same ticket within months and I had to spend hours doing the same research. This is because I didn’t take any notes when I received the original ticket. That’s why it is very important to take notes. When I start my research, I take notes as I look in the code or sql query. Sometimes I use notepad++ to take simple notes like text. For more detailed notes, I use Microsoft Word and take pictures. These pictures get annotated so we can focus on a specific area. No matter what program you use, take notes and share them with your team.

Transfer Knowledge

Just imagine keeping this knowledge to yourself. The internet was invented to share documents with other people. The same concept applies to support engineers. We have to share this information with other developers. Sharepoint or Confluence are tools that help software teams share important information. During pair programming sessions, take the time to share project architecture and any potential areas where the software can be made better.

When you take the role of a support engineer, you have to be familiar with the project. As you work on bugs, take notes so this knowledge gets share with other developers. Wikis are essential to transfer knowledge. Having a central place where information is shared will pay off dividends in the long run.

Categories
General

Dot Net Deployments – OSS Project to Handle .Net Deployments

I’ve been working on a new open source project called DotNetDeployments. Main purpose of this project is to learn .NET Core 2.0 and also to deploy any type of .NET project. Windows services, WCF services, ASP.NET MVC, Web API projects are handled with this tool. AWS CodeDeploy will be responsible for the actual deployment. PowerShell scripts will be used to setup IIS sites, install windows services, create back-ups and more.

I’m still seeing enterprise products being release manually or with a long list of instructions. The goal of this project is to handle all deployment activities with this tool. Sometimes we don’t have the budget to invest in deployment tools like Octopus DeployAppVeyor, or TeamCity. This is a new tool that will help those organizations automate 100% of their deployment activities and reduce human errors.

How to Contribute

Since .Net Core runs on Windows, Mac OS, and linux, you can use any text editor of your choice. I recommend using Visual Studio Code as your editor. First thing you have to do is clone the repository. Next commit your changes and once you are satisfied, open a pull request. I’ll try to respond as quickly as possible.

To be able to test your changes against AWS CodeDeploy, you have to setup a profile on your local computer. You have to name it “dotnetdeployments-profile”. Take a look at the official AWS doc online for a full explanation on credentials/profile setup.

In the next days, I’m going to create GitHub issues to manage all work needed to make this tool successful.

Categories
General

How to Clone your Environment?

We have git clone repository. After this command, you will have a ready to use git repository. What do we need to do to clone an entire environment? In our case, an environment consists of databases (SQL Server, MongoDB, PostgreSQL, MySQL), web sites, web services, and windows services. We currently have development, test, and production environments. Most of the settings live in a configuration project. When we build our applications, we have to specify a configuration (Dev, Test, Prod) setting.

With this setup, it is difficult to create new environments. We would have to add new config settings and add connection strings, S3 buckets, app settings, rest api urls, etc. One possible solution is to host all settings in a database table and link the servers to this environment. But now, we have to change our code to use the new object with all our settings.

I envisioned something like: var customerDatabase = Settings[“customerDatabase”]. There is no link to a specific environment. Everything is managed at the database level. And all settings are encrypted by default.

Cloning an environment is extremely difficult using on-premise servers. It is possible but it would take weeks and many meetings. On the other hand, using a cloud provider opens up our possibilities. Using AWS, we could create images from production instances and use those images to create a brand new environment. There are also cloud formation templates that could helps us create new environments.

What about deploying our applications?

We could use AWS code deploy to handle it. Once we have a VPC, servers, databases in place, it is a matter of deploying our applications to the right server. If this is a new environment needed to test a new feature for example, we could setup a script to turn-off instances after business hours. It would be available on demand only saving on cost. Your CFO will be happy to know your are saving money.

Let me know how you clone environments. Thanks.