His entire site is chock full of fun little features from random call-outs to even funny little Editor’s Notes in the articles:

He also hits on a topic that I didn’t expect from a design blog:

What To Say When You Don’t Know What To Say .

Growing up with ADHD & the anxiety that came with it I tended to react to things like grief by avoiding the even mention of it. Not only did I not know how to express my own feelings as a child but that also meant I felt like I was also unable to comfort others because of a self-imposed cycle of self-doubt. I was always second-guessing myself and was terrified of saying the wrong thing to someone I knew already was hurting.

What do you say to someone you know is in pain? To someone who just lost someone they love? It’s not an easy topic and sometimes there is no “right” answer, but as Henry so eloquently writes in his post, there are some things that should probably be avoided. Having been on the receiving end of at least one of his examples I have to say this post hit me in the feels and I definitely was not expecting that when I first was exploring his site. Instead of taking away from the well-written post that Henry wrote I will instead share with you one thing that will always be a safe way to convey that you care - “I’m sorry for your loss.”

Sometimes the less that is said can mean the most unspoken.

Now go check out his article then for a mood booster click on any of his blogroll or portfolio links as it seems most of his friends are also pretty good at web design.

So like him, I’m not a big fan of meta posts. Most of the time it just reeks of a faked interest to drive up site engagement but if you’re familiar with some of Kev’s work you’d know he’s pretty genuine about his intentions and a big believer in community building/indie-web. So he’s adopted a similar blog question challenge that he came across on another blogger you should also be following (Brandon’s Journal ) called the Bear Blog Question Challenge. Also not being in the BearBlog network (which btw is a great micro-blogging system if you’re in the market for one) I figured I’d join him in answering his questions on here. So here we go:

My Answers To The Challenge:

Why did you start blogging in the first place?

I was brought up around the time the internet was becoming more and more accessible on the consumer level and was lucky enough to have access to it while still in middle school. I think I was using MySpace and a few sites like Geocities to post random stuff I’d come across from alternative uses for AOL CDs to strategy guides for the games I was playing to really just anything (lots of song lyrics and what I thought they meant type posts). I didn’t actually start a ‘real’ blog until probably 2006 after graduating college. I was working in the advertising industry and had a lot of opinions on what was going on, ideas of what would be cool to see, etc.. I had taken on the online handle Brentter (aka my first name and the first three letters of my last name) a year or two earlier so naming it was easy - Everythings Better With Brentter. That quickly turned into not only a personal blog but an advertising industry blog as I was posting sometimes 3-4 times a day with campaigns I’d find or even be sent directly by production companies.

What platform are you using to manage your blog and why did you choose it?

This site is currently using Hugo and I chose it because I’d read nothing but great things about the speed at which huge Hugo sites could load. By the way it’s 100% true. It’s insanely fast not only to load the pages (because they’re flat files, not dynamically generated every time a user visits the page) as well as incredibly fast to compile the site when you’re ready to push an update to it. I’d been messing around with the go language for the past 2-3 years and that’s what Hugo is written in so I thought why not?

Have you blogged on other platforms before?

Yes. So prior to my first ‘real’ blog those platforms were mainly HTML/CSS based pages with no real structured CMS system, then with brentter.com it was initially built on some wiki software which for the life of me I can’t remember. That lasted about a year then I switched it over to Wordpress, which I had heard was just a new fork of another CMS named b2 (that I liked). Over the years of running that blog I’d dabble in little side projects using blogger and tumblr but ultimately stuck with Wordpress until I shut the site down somewhere around 2016/2017.

How do you write your posts? For example, in a local editing tool, or in a panel/dashboard that’s part of your blog?

So unlike my experiences with Wordpress, where I would either draft them in the WP back-end or use a 3rd party application that would help me manage the blog content better (Using the xml-rpc protocol), I write my posts using a great app called Sublime Text. I’ve been a fan and paid user of this application since it first came out and it’s only gotten better over the years. Hugo relies on pages or posts having what’s called front-matter. It’s a bit that goes at the top of everything you want to add to your Hugo site which indicates all the meta data as well as instructions on how my settings on the platform should render the page. So I have a couple of pre-made templates that I pull up in Sublime and that let’s me pretty quickly get working on something new. It’s a pretty easy to use setup. I then use git to push it to GitHub which then in turn triggers it being compiled and sent off to CloudFlare where I host brentter.com.

When do you feel most inspired to write?

Usually after reading or coming across something interesting, or at least something I think others would find as interesting as I do. So for instance I’m writing this probably 20 minutes after having read the post on Kevs blog. Though in this case it’s not because I think anyone cares about how this site is made or my thought process behind it, but rather because I like the idea, enjoy Kevs blog and want to participate. This is one of those few “this is for me” type posts. I have a bunch of post ideas lined up with a bunch of research that was needed to properly explain the content but those aren’t quite ready yet. Those got started right after getting the idea for them. I’ve got ADHD so the faster I act on an idea that I like the better chances of it seeing the light of day.

Do you publish immediately after writing, or do you let it simmer a bit as a draft?

Well this time around for my blog on this domain I tend to just publish immediately after double checking images, grammar, links, etc.. which leads me to another great feature of using Hugo is it has a built-in dev server so as I save changes I can view my updates in real-time before pushing live. On the old site I would tend to save the posts as drafts then give them a once-over before going to bed in case my tone was too strong in a negative direction when talking about other production companies or ad agencies. I didn’t want to unnecessarily burn bridges with a wild hot take and by then I’d usually have another idea or two to add.

What’s your favorite post on your blog?

So this version of the blog is still very new so I don’t have that many posts yet. In fact, I don’t really have any of my older style of posts that were long-form and tended to spend significant time arguing whatever my current point of view on a subject was. That being said I’d have to go with my post on adding a recently played music page to a Hugo blog. I threw it and the code together pretty quickly because it was something I wanted to add on here then quickly re-wrote the entire thing about a week or two later to incorporate it into a much bigger project I have yet to release. That’ll come out soon enough but basically it’s a homegrown scrobbler to track not only recent music but also bookmarks, starred reps, notes, exercise logs, movies, etc.. Writing that post is what gave me the idea for it and I’m pretty happy with the result no matter how basic the approach to it was. It goes back to something a favorite teacher once told me that “Success is not about the results from a first attempt, it’s about the progress you make after it.”

Any future plans for your blog? Maybe a redesign, a move to another platform, or adding a new feature?

So this being my first foray into using Hugo I went with a pretty basic theme that by now has been modified drastically for various features behind the scenes. While I like it, I know I can do better so I have been working on a new design using what I’ve learned about how the Hugo framework operates. An obvious new addition will be an email to reply button as mentioned earlier in this post. I also plan on having it incorporate more of my data to expand this from being just a blog to more of a personal info-garden.

So that’s it. Simple enough, while I don’t have anyone in particular in mind to tag (which is know is part of the whole concept of these meta-posts), I hope you go and check out both Kevs and Brandons blogs which are linked above. If you’re interested in joining us in this little mini blog challenge you can find more about it on Kevs Blog Questions Challenge page.

Cheers,
Brent

Raindrop And The Lack Of A Recently Added RSS Feed

The first thing I did was ask in their forum about it and apparently it is something that’s been requested numerous times over the years with the developer giving the same response saying that feature isn’t implemented because of privacy concerns. He goes on to say that each collection that you create has both a public and private RSS feed but the private one has a “uniq cryptic url” (his wording) and if shared will give full read access to that content. It’s not opensource so I have no idea how he’s set up the service but I’d have to assume that changing the method that the developer currently has in place for RSS feeds would be too much of a time-sink to be worthwhile for him. Raindrop has been getting a lot of positive press over the past few years not to mention a massive influx of new customers so why should he spend all that time just for a feature to appease a small subset of users? The developer has mentioned in a few of the RSS-related threads that Raindrop works well with glue-up services like IFTTT but involving another 3rd party service just isn’t what I was looking for either. As my mother would say, “I’m not mad, I’m just disappointed.” Sadly I discovered this problem after I had already signed up as a paid subscriber and then immediately regretted my decision.

My personal opinion on anything that’s online is you can never provide enough RSS feeds or ways to export your own content. If I create something online using your service or application (in this example a bookmarked link with my own tags and comments regarding it), then I should have an easy way to re-purpose and export that content from your site. This goes 20x if I’m paying for the service or have purchased the application. This opens up a much larger issue I will eventually talk about regarding a HUGE problem we’re seeing today with people not realizing that they don’t own anything that’s online unless they’re the ones in charge of it. You never know when company policy could change, the site could go offline, the company could go bankrupt/be sold, or any other of the million situations where the end result is you no longer have access to your data.

Export/Backup Options And The Raindrop.io API

Although Raindrop lacks an RSS feed for recent account activity they do incorporate a few other ways that you can download your data. You’ll need to be a paid subscriber to use these two features but they have a export/backup feature that includes options for it to automatically save regular backups of your account to various cloud services (like Dropbox) as well as an API. I went with the API option and wrote a quick little python wrapper for it. It’s pretty basic but should work with authenticating, reading/editing/deleting raindrops and grabbing collections. If you wish it could work with any of the other API functions feel free to submit a ticket on GitHub and I might add it for you. You can find it here - Github.com/Brentter/Raindrop.io-Python-API-Wrap

I also included in that repository a basic python script to download the most recent 25 bookmarks from your account, remove any unnecessary or redundant data that’s included and then save it locally as a .json file. Back when I put that all together I was doing something similar for a couple of different services with an additional script that would run every few hours to see if any of them needed updating, trigger those scripts then add anything new to a local database. I like keeping functions as separate as possible so they’re easier to change (and scale). This also allowed me to add it to a self-hosted “recently saved content” page that wasn’t reliant on outside sources to use (only to update). For my own projects I’m very much a ‘functionality over design’ kind of guy so I won’t be sharing that part of the project anytime soon (it’s not pretty) however you can easily get similar results using any locally hosted RSS reader.

The Download Recently Added Bookmarks Python Script

Here’s the basic download script using my Raindrop.io API wrapper in python, you’ll need to add your own token from the app to a .env file before running it (full instructions on the git page posted above):

from raindrop_api import RaindropAPI
import requests
import json
import argparse
# You can change the number of results it will pull with -l or --limit - Default is 25

def get_recent_raindrops(api, limit=25):
    """Fetch the most recent raindrops."""
    url = f"{api.base_url}/raindrops/0?perpage={limit}&sort=-created"
    response = requests.get(url, headers=api.headers)
    return response.json()

def filter_raindrops(raindrops):
    """Filter the raindrops to only include specific fields."""
    filtered = []
    for item in raindrops['items']:
        filtered.append({
            'link': item.get('link'),
            'title': item.get('title'),
            'excerpt': item.get('excerpt'),
            'cover': item.get('cover'),
            'tags': item.get('tags'),
            'created': item.get('created')
        })
    return filtered

def main():
    parser = argparse.ArgumentParser(description='Fetch and filter recent raindrops.')
    parser.add_argument('-l', '--limit', type=int, default=25, help='Number of recent raindrops to fetch')
    args = parser.parse_args()
    
    # Initialize the RaindropAPI
    api = RaindropAPI()
    
    # Get the most recent raindrops
    recent_raindrops = get_recent_raindrops(api, limit=args.limit)
    
    # Filter the raindrops
    filtered_raindrops = filter_raindrops(recent_raindrops)
    
    # Print the filtered raindrops in JSON format
    print(json.dumps(filtered_raindrops, indent=4))
    
    # Save the filtered raindrops to a file in JSON format
    with open('recentraindrops.json', 'w') as file:
        json.dump(filtered_raindrops, file, indent=4)

if __name__ == "__main__":
    main()

So when using their API it can give you a bunch of data fields that I didn’t have a use for, hence the quick filtering job. Feel free to use this and the API wrapper for whatever purpose you want. If it’s helpful I’d love to hear about what you made!

Cheers,
Brent

Combining Git Repositories Into A Single File - GitIngest

Some of the recent AI programming assistants have had mixed results when faced with learning from entire code repositories and one idea that’s been gaining traction is to combine the files into a singular source so that it “hopefully” won’t get confused as to how a piece of software or app works. One of the easiest tools I’ve come across is an open source project by Romain Courtois called GitIngest . You can use the online app over at GitIngest.com , self-host it via the provided docker containers, run it locally or even use it as a python library in your own projects. It comes with it’s own simple front-end, ability to restrict allowed hostnames, ignore file-types, include files only under a certain size, basic token estimation and it even lets you preview the output before downloading the final compiled file. Pretty useful if you ask me.

Converting a Website Into Markdown Content

So there are a bunch of tools out there that will let you convert single pages and even entire websites into a standardized Markdown format as well as few open-source projects that can help you write your own tools to do the same. Let’s break down the basic process behind it:

• First you’re going to need to scrape the webpage.
• Next you’re going to need to parse out all the information you don’t want or need like ads, tracking scripts, headers/footers and if possible most of the navigational UI.
• Once you’ve done that then you need to convert that content into Markdown.

While the basics behind the format are pretty standardized there are a few popular variations you will come across like MultiMarkdown, Markdown Extra, Pandoc and GFM (Github Flavored Markdown) in addition to some sites like Reddit and StackOverflow that use site-specific versions of it for their own functionality.

Tools I’ve Used To Convert Websites to Markdown For LLM Use

Here’s a couple free tools, most that you can host yourself, that will help. I’ve used all of them at some point in the last year and as I’ll get into, some work better than others for certain kinds of content/websites.

MarkdownDown - This is a great tool, and the one I’ve lately been using the most, to scrape and convert websites into Markdown. Asad Memon has it hosted for free to use on Vercel as well as the full source code on his Github. It uses Puppeteer to scrape the page, Turndown to convert it to the Markdown format and the Mozilla Readability library to clean up the content of any unnecessary un-desired crap. It has the option to add in a Browserless API key so you don’t have to scrape it yourself as well as includes the option to spin up a Cloudflare worker who uses the Browser Rendering API (which requires the $5/month paid Cloudflare Worker plan) to do the same. It has a nice front-end that let’s you incorporate your own OpenAI key if you want to have it manipulate the content before outputting it to markdown as well as the option to download the images (and re-link them) locally. If you’re trying to scrape pages like reddit there is currently a working pull-request by laggingreflex which enables the puppeteer-extra stealth plugin to circumvent blocks.

HeckYesMarkdown.com - Created by Brett Terpstra , heckyesmarkdown will scrape a provided url and output the contents in a variety of customizable formats ranging from multiple types of Markdown to HTML, LaTeX, OpenDocument, Plaintext, RTF and more than a dozen others. It provides a convenient bookmarklet you can put in your browser to use it on the go and displays the results in an easy to copy format with the ability to preview what the rendered markdown will look like in a browser. It also let’s you import the result directly into apps like Obsidian (which I love). He also was kind enough to provide an API in case you wanted to incorporate the service in your own scripts.

URLtoMarkdown.com - This is a great open-source tool using Turndown, Readability, jsdom and Heroku to convert a URL into markdown and allows you to ignore links and/or include the page title in the result. You can use the tool on their homepage or go to their Github to host your own. One of the best features of this site is it also provides a web service that will convert a URL to markdown simply by adding it to the end of their Heroku app (that you can host yourself)- i.e. https://urltomarkdown.herokuapp.com/?url=https://example.com

Jina.AI Reader API - Jina.AI provides a free reader API and web-service that can convert URL’s into formats like Markdown, TXT, HTML and even a screenshot. While you don’t need an API key to use it, you are rate-limited by them if you don’t. Luckily they provide a free API key with 1 million credits on it (yes, 1 million) just by visiting their website (no registration/login/credit-card required). I hope they don’t read this but it’s also just as easy to get a new one once you’ve used that up just by clearing your cookies and revisiting the same page. It let’s you grab the entire website, select a specific CSS target to include or exclude, remove all the images, gather all the links and/or images at the end, forward custom cookies, use a proxy service and a gaggle of other useful options for hard-to-scrape sources like Reddit or dynamic javascript-based sites. With that same API key you can also use a few of their other API-based products like their AI re-ranker, zero-shot classification tool and more. To use their free web-service just add your desired URL at the end of r.jina.ai like this: https://r.jina.ai/https://example.com

Firecrawl.dev - Firecrawl is a pretty popular tool that offers both an open-source model that you can self-host as well as a fully-featured website API tool that is designed to scrape, crawl and extract entire websites or even libraries and convert them into LLM-ready structured data. You can save the content as Markdown, HTML, screenshots or use their custom Extract LLM service to export it in LLM-specific formats (still in beta). It’s very fast, works on most if not all websites I’ve thrown at it and if you don’t want to self-host it they provide you with 500 credits for free to test it out. Firecrawl is used in a LOT of the big frameworks that you’ll find out there and some tend to lean on it pretty hard as their go-to LLM data rendering/scraping/extraction service. If you’d rather just pay a small fee to have everything taken care of for you in an easy, ready-made platform this would probably be the one to look into. They have an endless list of features and integrations, incorporate all sorts of different media parsing into their runs and allow you to set up crawl/scrape jobs without needing much technical knowledge. They outline the big differences between their open-source offering and their paid cloud service on their docs page here .

Webscrapbook - Webscrapbook is an opensource browser extension available for both Chromium-based browsers (Chrome , Edge , Opera, Vivaldi, etc..) as well as Firefox-based browsers (Firefox , Tor, Waterfox) that let’s you save any website in a variety of formats as well as manipulate and notate the content before saving it. You can also have it save the content both locally on your own device or remotely so it can be easily shared with others. Mobile friendly, backwards compatible with previous legacy versions of Scrapbook X and offers the ability to take your saved pages and generate a static site index that can be used as it’s own static website (which can come in handy when ripping data for purposes other than LLM-use).

If you search on GitHub or even google for “website to structured data converter” you can see there is an endless supply of both paid and open-source tools available that all have their own pros/cons. Using a combination of the ones above I’ve haven’t found any source that I wasn’t able to easily convert to markdown. Hope that helps.

Cheers,
Brent

So I’ve been a big supporter of syndication protocols like RSS/Atom ever since I first stared blogging 20 years ago. It just makes curating content/information super easy. I’m not going to get into all the different options out there for you to manage, filter and manipulate your chosen set of feeds because it’s a topic I love to talk about and quite frankly could easily be here for weeks turning this into a 20 part 5k word each posting series so instead I’ll just name a few of my favorite free ones. For self-hosting your own RSS aggregator it’s really hard to do better than FreshRSS or Miniflux . Both have huge support communities, are open-source, easy to install and use and can handle hundreds of thousands of feeds with minimal system resources.

For using an actual RSS reader application that you can download on your desktop or mobile device I have to go with my new favorite app Follow.is followed by Netnewswire or Raven . Again, all are free, have good support communities behind them and universally work well regardless of your environment/use-case. I’m a big fan of Follow because it’s actively being worked on by a team of dedicated developers and includes a bunch of features that you have to pay for to see in other RSS readers (like AI integration, custom actions and filter options, RSSHub integration, etc..). The question I always get asked though is how do I find RSS feeds that are worth subscribing to? There’s just so much content out there that sadly it’s pretty difficult to filter through all the noise to find what matches what your interests are.

Feedle - RSS Feed and Podcast Search Engine

Feedle - A RSS and podcast search engine that does a pretty good job of helping you filter out and discover content sources that are actually about whatever your search term is. Here’s an example, let’s say that you’re like me and enjoy rock climbing in your spare time so you want to read about it when you’re stuck on the train headed to work. Here’s the search results for rock-climbing - It’s pretty spot on as far as the quality of the posts. Searching a site like google for the same query will bring you nothing but ads and 100% AI generated BS. I like how Feedle displays the search results too - you not only have the basics of the post image, description and direct link but it also includes any associated mastodon links for the author. Two key features is Feedle allows you to filter out any of the content sources you don’t want to see in your results as well it will turn any search query into its very own RSS feed! You can then just subscribe to the search query RSS feed and stay up to date with that topic without the pain of having to curate each source for just posts related to what you’re looking for. I can’t tell you how helpful that feature is when you’re just trying to stay up to date with whatever the latest in a new technology is. If you don’t have a specific topic in mind you can also click on their Top Stories link to see what other people have been recently been hyping up online.

Feedle also has a few of their very own curated categories that they provide. Two of my favorites are Good-News which are just positive stories from around the internet and Digital Crossroads which are a mix of updates and news related to the open web/tech.

There’s still a few alternative search engines that can help you find content sources like Teclis , which was created by the CEO of Kagi and only includes non-commercial sources. It’s not really “feed-focused” though as it acts like a regular search engine just with a filtered index of sites to pull from. For an expansive list of independent/unique search indexes check out this blog post from 2021 on Seirdy.com . Happy hunting!

So over the years I’ve been a pretty faithful Scrobbler of the music that I listen to so that I can not only keep track of new songs or artists that I come across but also because it makes sharing that content a bit easier with others. The problem I’ve run into though is how those links look when embedded on different social platforms. For instance a link to a song that is hosted on YouTube will look one way when posted on mastodon and a different way when posted on twitter. In the past I’d just chalk it up to the cost of doing business on multiple social networks but now that is a thing of the past! Let me show you this completely free & open source music poster generator that I found on Github named BeatPrints . It’s written in Python and can be used for an individual song or for the entire album. Here’s two examples that I made from earlier today: Vincent Neil Emerson’s song Manhattan Island Serenade is on the left and is using the dark theme with the extra color flair option selected and Sierra Ferrell’s cover of Years is on the right using the light theme as well as the extra color flair option.

In addition to letting you choose between the light and dark themes it also allows for custom images, the addition or removal of that colorful bar in the middle (that’s the extra flair option) and custom lyrics. Installing it on any platform is a cakewalk and setting up the config file is just copy and pasting in your Spotify API key and deciding where you want it to save your images. After it finds what song and album you want to showcase it will then search on Spotify for the lyrics. If it can’t find any it will still give you the option to copy and paste in your own. Now I’ve down-scaled and optimized the two images you see above but the original files that the script creates are all high quality .PNG files.

Anyhow, it’s just a cool new way to showoff your music library. If you’d like something similar but without having to install anything, there’s a web-based poster generator that you can also customize a bit called Posterfy .

Here is a much needed update on the latest all-in-one RSS tool Follow that I first wrote about last September. Since then a bunch of features have been added and the user-base has been increasing like wildfire. They’ve added local Obsidian vault integration, the ability to add and view email newsletters, a custom tool to create new RSS feeds for websites or apps that don’t currently have one and the addition of external webhooks to your custom Actions. In addition to that there’s been a bit of discord shenanigans involving the sharing of invite codes that I’ll talk about later on. Without a doubt though the biggest update is that Follow is no longer invite only! Anyone can now join without requiring an invite code to use the app. Just head on over to the follow.is website and sign up. The catch is while they’re still rolling out new features, you’re limited in the number of feeds and lists you can add to your account unless you get a user-generated code that unlocks everything. It’s the same user-generated code that previously were used for invites so any user who was invited in the past or has already entered a code can generate a new one that they can share every five days (or even sooner if they complete accomplishments).

Create RSS Feeds For Any Site or App

Now you can take sites or apps that don’t have a native RSS feed integration and Follow will extract from it the necessary data based on CSS attributes to create an updated feed you can then not only follow but share with others.

Email Newsletter Integration

If there are certain email-only newsletters you would like to stay on top of you can now give it your very own Follow email address and it will add them to your feed list. Here’s an example using the weekly console.dev Developer Tool & Betas newsletter (it comes out every Thursday and is a nice little heads up on new sites and tools out there).

Actions now with 25% more Action!

Previously I showed how you can use their Actions function to automatically translate feeds to the language of your choice, well now they’ve added the ability for it to trigger outside webhooks. That gives you the opportunity to integrate it into your own application/system whether it’s responding to someone mentioning you or your product or even just adding the user to a list of potential customers via your CRM. It’s perfect for using with automation services like n8n, Zapier or IFTTT. Currently I still have it translating certain social feeds (the developers of Follow are primarily Chinese and I like to keep up with their development notes) as well as summarizing certain blogs that can tend to be a bit verbose in length. I probably will fool around with this feature a little more and maybe even set up a webhook to auto-respond to people who link my website on social media with a “Thanks for sharing” type message.

Save to Obsidian Integration

For Obsidian users like myself you can now enable the option to save content from Follow directly to your Obsidian vault just by setting up your vault location under preferences and clicking the “Save to Obsidian” button while viewing content.

Invitation Code Puzzles

A curious thing started happening with the Follow app since my first post about it. While still in alpha the discord became flooded with requests for invite keys and as a result, certain users decided to have some fun with it and created puzzles of all sorts that once solved, would reveal a fresh key. That quickly caught on and has become an almost daily trend with users posting new puzzles for others to solve. They even created it’s own discord channel specifically for code-puzzles. Here’s a few that stood out:

• A user named Tony has been creating MD5 Hash related puzzles for new codes that seem to keep escalating in difficulty with each attempt.
  • 
  • The answer for this next one was (gvcmCSYp_V)
    
  • Here’s one where the winner said it only took 30 seconds to solve using hashcat on his 4090 (answer was f2ZCTi_2LV in case you were curious)
    
• A user named Cap_stu posted two different steganography based puzzles, this first one I think took the discord the longest out of all the puzzles to finally solve:
  
  • This one was another from cap_stu after it had been solved because so many people wanted to know how it was done. It gives you an example of the amount of effort people have been putting into creating these.
    
• A user named Zvezda posted a txt file containing a string of 10000 characters, all 1s and 0s. To solve it you had to divide it into 100 groups of equal characters then when arranged top to bottom with 0 representing black and 1 representing white a QR code with the invite code would appear.
  
  
• Another involved decoding a string through multiple base* decodings best solved using a tool like cyberchef (string->decode base45->decode base58->decode base85->decode base92->answer)
  

Follow Is No Longer Invite Only!

Follow no longer requires an invite code to be able to join and use the service however it does require one to unlock all the features. Basically you can check out what it is capable of but are limited to only a small number of feeds and lists. Any invite code from a user unlocks it to its full potential so there are still puzzles being posted regularly in the discord. If you’re interested feel free to join their discord or visit their website at follow.is . I currently maintain two lists on their service (think of them like OPML feeds) that make keeping up with security and homelab news easy that can be found here:
Homelab Resources - Host your own anything!
and
Security Updates/Blogs

Give them a follow (pun intended) if you’re interested in either topic or just follow me directly here: brentter@follow

Venv and Virtualenv are two of the most commonly used virtual environments for the Python language. Both have been integrated into the Python core since version 3.3 so using them is pretty much a piece of cake. While there is a difference between the two, they actually are made to work together. Which one you’ll use all depends on what your needs are. Do you need to use a different version of python for that one project that you don’t use anywhere else or do you just want to keep dependencies in sync?

Why Use a Virtual Environment for Python?

So I’ve been asked this a few times in the past by folks who read in various guides to always use a virtual environment with python, especially if it’s something you’ve downloaded off from GitHub. Now that advice doesn’t have the same magnitude of outcomes as when you were told to always wear a rubber but it can still cause some major frustration and headaches if not followed. The reason to always use a virtual environment is to keep both the project and your working environment synced up in terms of version control. Many times you’ll find code you’d like to implement on your own system but it uses deprecated or out of date dependencies and either you have to go through the hassle of downgrading everything, update the code to work with what you have, or start over. There are a couple different virtual environments but honestly the two easiest to implement are Venv and Virtualenv.

Venv - The Built-in Python Virtual Environment

So venv is a virtual environment that’s been built-into Python since version 3.3. So unless you’re using a very outdated setup you should already have it installed. It allows you to create an isolated Python environment using the current version of python you have installed. Each environment uses it’s own Python binary and an independent set of Python packages. It’s pretty fast and easy to use and setup and allows you to keep projects of all different types in-sync and usable. Here’s how to use it:

Create your new virtual environment by going to the root directory of your project and issuing the following command (i’m assuming you’re using some form of linux):

python3 -m venv .myproject

This will create a virtual environment in a new folder named .myproject (rename to whatever you want).

Now it’s installed, we need to activate it for it to work. Do that by using:

source .myproject/bin/activate

Your prompt will usually have a new character/notification on it when you’re in a virtual environment (at least in the majority of terminal setups) but you can always test it by typing

which python

It should return with the current file-path with .myproject/bin/python at the end

Now you can install whatever packages you want and it won’t affect the rest of your system. Easiest way to do that is with pip. Here are a few examples:

python3 -m pip install requests

#for specific versions of packages:
python3 -m pip install 'requests=2.11.2'

#from a local archive
python3 -m pip install requests-2.24.1.tar.gz

#if you're in the directory of the source code for a package you need to install use:
python3 -m pip install .

#If there's a requirements.txt file you can install all of those with:
python3 -m pip install -r requirements.txt

To export a list of all installed packages for that current virtual environment you can use freeze and it will show you both the package and current version:

python3 -m pip freeze

#or to have it write it's own requirements.txt file:

python3 -m pip freeze > requirements.txt 

to deactivate it when you’re done all you have to type is:

deactivate

Virtualenv Python Virtual Environment

Virtualenv is similar to venv, also standard in python since version 3.3, but the main difference is it allows you to create a virtual environment using different versions of Python. It works by being installed as a subset of the actual venv command so once installed it’s usage is almost identical except in how you create your environment.

First install virtualenv with pip:

pip install virtualenv

Now let’s go to the root folder of our project and run:

python(version) -m venv my-virtual-environment-name-here

#so like this:

python3.11 -m venv myproject

Now just like with venv it will create a virtual environment in your folder under .myproject/bin/

Activate it (to start it) with:

source .myproject/bin/activate

Installing dependencies is the same as it was with venv so for a bunch of examples just scroll up to the previous section.

End it by using deactivate:

deactivate

Notes

Make sure that regardless of what virtual environment you end up using to always have your .venv (or in our example: .myproject/) folder added to your .gitignore file. Otherwise your repo will download all the python files with it.

Burger King’s launch of their new Angry Whopper in 2009 changed social media advertising forever. They’d already shown they could pull off a popular viral website idea with The Subservient Chicken but what they did next for The Angry Whopper was nothing short of insanity when you realize it was on behalf of a fortune 100 client. This campaign showed there was nowhere online you could go where advertisers wouldn’t be pulling out all the stops to get you to engage.

Burger King’s Angry Whopper

The original version of the Angry whopper sold back when it was first released in 2009 was just your regular Whopper but with spicy onions, jalapeños, pepper jack cheese and a spicy “Angry Sauce.” As with everything from that era of Crispin Porter + Bogusky (just Crispin now) it had to be pulled off in an extreme way. For 2009 that meant the launch of both a website (which is long dead), angry-gram.com as well a Facebook campaign that had them temporarily banned from the platform.

The Facebook Debauchery

This Facebook campaign required you to unfriend 10 people through their app (who would be notified that you unfriended them for a burger) and they would reward you with a coupon for a free Angry Whopper. It was so popular that it scared the higher-ups at Facebook and they forcibly took it down after only 10 days for violating it’s user agreement. The app alone received 13 million media impressions while it was active and jumped to 32 million after it had just a landing page guiding you to the angry-gram website.

Over 230k people had been unfriended and it was more popular as a taboo, now deleted app than it was when it was actually live. They were the first advertising agency to figure out how to milk that platform for views and that was by showing just how little people cared about who was on their friend-list and how easily you could game the system. Before this they had introduced us to how to create an internet phenomenon with The Subservient Chicken with the site gaining almost 400 million hits within a year (average time spent per user on site was 7-8 minutes!). They also created a few other off-color online properties like WhopperFreakout and WhopperVirgins, both sites getting a quarter of a million views each in the first month alone(both sites are also dead so no link). They broke the mold on how to gain consumer engagement online.

The Google Ad Campaign

Creating games and fun commercials wasn’t the whole campaign though. They also purchased top ranking positions for various keywords related to not only the Angry Whopper but also numerous other high-priced search phrases and competitor products. They undoubtedly were having to increase their ad buy daily as competitors moved to try and outrank the BK ads. The combined strategy left you very few places on the internet where you wouldn’t come across some sort of Burger King related media. This was probably one of the more expensive PPC buys for any brand at the time, let alone a fast-food restaurant.

My Initial Reaction

When I first wrote about this back in 2009 I said the following towards the end of my little write-up:

It combines word-of-mouth marketing with social media in a way that encourages users to spread the application across their online communities. It’ll be interesting to see the analytics on how well this application does in a month or so.

A couple of months later I remember talking with a good friend about the metrics/analytics they reported and he put it perfectly, “If Bogusky comes to you with an idea, sign off on it immediately.” Boy was that the truth. By the time of this campaign he’d already been inducted to the American Advertising Federations Hall of Achievement , received an honorary doctorate from the University of Colorado Boulder and was crowned the Creative Director of the Decade by Adweek (In addition to winning just about every award out there in the industry). What was left to do? Easy, he “retired” from advertising and moved to Boulder, CO to become a consumer advocate in 2010. He’d eventually return to CP+B for a short stint in 2018 only to retire yet again in 2020. Nowadays you can occasionally catch him on the “Joy of Bike ” YouTube channel helping connect kids with bikes & teaching us “Over 40s” how to ride fast. Go subscribe and if you get a chance to hear him speak, I’d recommend it as he has a fascinating take on life.

A Not So Spicy End

They launched the Angry Whopper internationally with various TV commercials to back up how uncontrollably spicy it supposedly was. I remember trying the burger back when they first came out because I’m a big fan of spicy food and may have eagerly dropped ten friends to get one for free (I can’t recall). I’m sorry to report though, this was quite literally a nothing-burger in terms of how spicy it was. After the initial media blitz and charm wore off and negative reports started coming in via food reviewers they eventually took it off the menu. At least, for a few years that is.

Attempts at Bringing The Angry Whopper Back

Burger King has attempted to bring back the Angry Whopper multiple times throughout the past decade, notably in 2012, 2013, 2016 and 2019 all featuring their own over-the-top commercials. While they never achieved as great of a social “buzz” as they did with the first campaign in 2009, they did manage to create a mini-cult following . While I wouldn’t be surprised if that was actually just another brand-sponsored astoturf campaign you have to admire their tenacity. They even tried renaming it to the Angriest Whooper, which also didn’t pan so well in terms of taste and consumer reception.

Is The Angry Whopper Ever Coming Back?

All signs point to yes. They recently brought it back late last 2023 but only in the Columbus, Ohio area and some parts of Canada. In case you were unaware, Columbus is ground zero for fast food companies testing new products. A mixture of the large size of the city added with the fact it’s right in the center of the Midwest, then you top that off with the prolific number of colleges in the area and you’ll understand why.

Who Was the Dos Equis Most Interesting Man In the World?

In 2006 ad agency Euro RSCG/NYC (now Havas Worldwide ) launched an advertising campaign on behalf of the Cuauhtémoc Moctezuma Brewery for their Dos Equis brand named “The Most Interesting Man In The World.” A quick insert though - the history behind this brewery, which goes all the way back to 1890, is a fascinating read so I’ve linked their Wikipedia page and encourage you to check it out. This new campaign featured a handsome, grey-bearded actor named Jonathan Goldsmith who played the title role for almost a decade (ending with him boarding a space ship headed to Mars in 2016). The entire campaign was based around the epic adventures and pearls of wisdom of the man which the narrator, voiced by Will Lyman from Frontline fame, would describe always followed by Goldsmiths character delivering his now famous line, “I don’t always drink beer, But when I do, I prefer Dos Equis.” It would then feature a nod or toast from him and the new brand slogan “Stay thirsty, my friends.” Later in the campaign that last line would sometimes be changed to “Stay thirsty, mis amigos.” It was an wildly popular series leading to over a decade of TV and print ads in addition to hundreds of radio spots all featuring the classic Chuck Norris/Bill Brasky style of setup to further the lore of their new brand spokesman. Here are a few of my personal favorites from the radio series below:

“My words carry weight that would break a less interesting mans jaw”

“I once had an awkward moment, just to see how it feels.”

“If opportunity knocks, and I am not at home, opportunity waits.”

“My pillow is cool on BOTH sides.”

“I once received a standing ovation from a jurors box.”

A Little Info Behind the Initial Campaign

So this campaign was brought to the world by EURO RSCG/NewYork’s Creative Director at the time named Jeff Kling along with Steve Miller from Radical Media as the director. Outside Editorial was tasked with the post work for the first batch of ads. It was widely talked about, achieved a few awards over the years and apparently they had brainstormed over a thousand radio spots before narrowing them down to only the few hundred+ that actually made it on air. When they launched the campaign they also had a website aptly titled staythirstymyfriends.com (now offline) where you could “test your skill in patience and strength” by digitally arm-wrestling Churchill, Stalin, Mao or even the Most Interesting Man himself. It had a bunch of other funny on-brand features like a photo gallery and application for a job as his personal assistant. It was pretty clever, especially for 2006. They would continue on for almost ten years with this campaign until replacing Goldsmith with a much younger French actor named Augustin Legrand. While the campaign had definitely run its course by then, swapping out the actors resulted in another two or so years before we finally heard the last of him. Here was their intro commercial for Legrand, my favorite part was him running through the streets with a supermodel in one hand while cradling a pig wearing a fez hat in the other.

The Reaction From The Public

It’s crazy that this was even talked about so widely as it was back then but some people really seemed to feel like the beer brand had done Goldsmith a disservice by not ending the campaign when they sent him off to Mars. At least that’s what we’d hear on late night television and talk radio shows for weeks following the final commercial. All the drama surrounding the replacement would eventually lead to a series of high-profile interviews where the actor seemed to to just laugh about the whole ordeal and said it was time to move on. Even though he was no longer the official most interesting man in the world that doesn’t mean he’s no longer acting. From the looks of his IMDB page he still gets cameo roles in popular tv dramas, which honestly isn’t that bad of a gig.

So Why Where You Posting TV Ads On Brentter.com?

So at the time this was one of the few independent advertising blogs on the web and being in the industry myself I would spent countless hours in my spare time reaching out to other agencies and production houses with the promise that if they emailed me whatever upcoming campaigns they were working on I would make a post about it. Better yet, if they included the media content as well as the bio/list of those involved, I’d post it in it’s entirety. At the time, listing the entire group of people who worked on any given campaign was somewhat rare, as the advertising agency or just their Creative Director would typically take the credit as a whole. My pitch provided anyone that worked on the project, whether from the production agency that filmed and edited the ad to the copywriter from the ad agency that conceptualized it got credit. It was something they could link to their friends or family that would prove they actually worked on the piece. That ended up working pretty well and the Dos Equis - The Most Interesting Man In The World campaign was one of the emails I received with links to download the first few ads.

So What Happened To The Old Blog?

In those days I would host all the videos at this fairly new startup that featured unlimited storage for streaming videos. Then one day they finally emailed everyone to let us all know they’d been hiding their financial situation and that they were broke. Worse, they had been so late on their hosting bill that all their servers were finally taken offline. All my videos, aka my content, was now offline. That was big learning experience for me in many ways. First, don’t trust only one service to host everything that brings visitors (and money) to your site. Second, if there’s a free tier, that means you’re paying in some other way. If you’re not, then they’re losing in the worst of ways and you might one day find they’ve had their severs repossessed too. Third, always prepare for the worst with backup plans and backed up data.

So Why Revisit The Dos Equis Campaign Now?

With the amount of backlinks I had back then there was always going to be a source that was not actively check their own outgoing links for dead-ends (which everyone should do every now and then). That or the owners of the website had abandoned it or quite frankly could just be too lazy to care. Now I first noticed this the other week when I realized some of the old ad campaign pages were getting a few hits from sources that didn’t seem like they were bots. The only problem was they were all getting a 404 page because none of those links actually still exist. This post isn’t just a cheap seo hit-job though. If I wanted that I’d put the whole site back online. Back when I was writing all those articles I really did love the industry and still have a soft spot for creative marketing. In order to send any future visitors to the correct new URLs I set up a 301 redirect to this page and for the other links it will send them to my homepage. Backlinks and 301 redirects, still going strong all these years. If you are interested in learning a little bit more about how easy 301 redirects are to setup and what they can (or sometimes can’t) do for you in terms of boosting your website SEO, I actually wrote a post about it when I noticed all the old traffic. Check it out if you’re curious.

Until next time, Stay Thirsty My Friends.
~Brent

A 301 redirect is a permanent redirect from one URL to another. 301 is the HTTP status code given that tells browsers (and search engines) that this content is now located over at a new address. In addition to the notice it will reload the browser with the new URL. When a 301 redirect is implemented, it passes most of the link equity (ranking power on search engines) along with it to the new URL. At least most of the time that is the case. Now obviously this is something that comes in handy when moving to a new domain or swapping over your blog posts from one service to another but another popular usage of it is when websites with quality backlinks go for sale or their domains end up in auction. I talk a little more about that later on in this post.

Why Choose a 301 Redirect?

You should use a 301 redirect when you want to permanently move a page or an entire website to a new URL. It’s the best choice for SEO because it transfers most of the link equity to the new page. In contrast, a 302 redirect is temporary and doesn’t pass as much link equity. Choose a 301 redirect when:

• You’re permanently changing your URL structure.
• You’re moving to a new domain
• You’re consolidating multiple pages or websites

Easy 301 Redirects With Your Web Server

So for all the different sources you can use to implement a 301 redirect, I will show you the same three different example scenarios so you can get a good idea of what you need to customize for your own needs.

Nginx

For Nginx, you’ll need to modify your server block in the Nginx configuration file (usually located at /etc/nginx/nginx.conf or /etc/nginx/sites-available/default).

Scenario 1: Updating a single URL on the same domain

server {
    ...
    rewrite ^/blogone$ /blog/blogone permanent;
    ...
}

Scenario 2: Redirecting to a new domain

server {
    ...
    rewrite ^/blogone$ https://example22.com/blogone permanent;
    ...
}

Scenario 3: Redirecting multiple pages to another domain’s root

server {
    ...
    rewrite ^/blog/(.*)$ https://example22.com permanent;
    ...
}

After making any changes always test your nginx config files before reloading nginx:

nginx -t
sudo systemctl reload nginx

Apache

For Apache we’re going to be creating a file name ‘.htaccess’ and placing it in our websites root directory. Make sure it has the period in front of the name.

Scenario 1: Updating a single URL on the same domain

RewriteEngine On
RewriteRule ^blogone$ /blog/blogone [R=301,L]

Scenario 2: Redirecting to a new domain

RewriteEngine On
RewriteRule ^blogone$ https://example22.com/blogone [R=301,L]

Scenario 3: Redirecting multiple pages to another domain’s root

RewriteEngine On
RewriteRule ^blog/(.*)$ https://example22.com [R=301,L]

After making changes you might need to restart apache.

sudo systemctl restart apache2 

Easy 301 Redirects Using Programming

PHP

PHP redirects are server-side and can be implemented as follows:
Scenario 1: Updating a single URL

<?php
if ($_SERVER['REQUEST_URI'] === '/blogone') {
    header("HTTP/1.1 301 Moved Permanently");
    header("Location: /blog/blogone");
    exit();
}
?>

Scenario 2: Redirecting to a new domain

<?php
if ($_SERVER['REQUEST_URI'] === '/blogone') {
    header("HTTP/1.1 301 Moved Permanently");
    header("Location: https://example22.com/blogone");
    exit();
}
?>

Scenario 3: Redirecting multiple pages to another domain’s root

<?php
if (strpos($_SERVER['REQUEST_URI'], '/blog/') === 0) {
    header("HTTP/1.1 301 Moved Permanently");
    header("Location: https://example22.com");
    exit();
}
?>

Javascript

So I’m including this next one because I’ve seen this in practice before but it really isn’t going to be recommended unless it’s your only choice. This is because sites like Google claim their web-crawlers don’t read javascript.

Scenario 1: Updating a single URL on the same domain

<script>
if (window.location.pathname === '/blogone') {
    window.location.replace('/blog/blogone');
}
</script>

Scenario 2: Redirecting to a new domain

<script>
if (window.location.pathname === '/blogone') {
    window.location.replace('https://example22.com/blogone');
}
</script>

Scenario 3: Redirecting multiple pages to another domain’s root

<script>
if (window.location.pathname.startsWith('/blog/')) {
    window.location.replace('https://example22.com');
}
</script>

HTML Meta Refresh

So this is the old school way from back when you didn’t have access to making new files in the root directory or even seeing what the server config looked like. It’s not recommended if you have any other options but it will still work to an extent.

Here’s a few examples of the redirect, it’s pretty basic.

<meta http-equiv="refresh" content="0; URL='/blog/blogone'" />
<meta http-equiv="refresh" content="0; URL='https://example22.com/blogone'" />
<meta http-equiv="refresh" content="0; URL='https://example22.com'" />

Easy 301 Redirects To Your Blog/CMS

This is if you’d rather have your blog or CMS system handle it for you.

WordPress

Here are three completely free plugins that are all geared toward different types of 301 requirements.

Simple Website Redirection
This plugin was designed for one single purpose - to redirect the entire website (except the WordPress back-end) to somewhere else. You can also have it use a 302 redirect (temporary) to test your settings before switching it to a permanent 301.

Redirection
This is probably the most powerful of all the redirection plugins that are completely free. It was designed to be able to handle and remember thousands of different redirects and includes the ability to customize them all. From single pages to full directories it also has the ability to use regular expressions so you can create dynamic redirect patterns that it will follow. A cool feature on this one is it keeps track of every redirected visitor (ip/geo/referrer/header/browser) and allows you to also set redirection rules based on who is trying to access that page. Redirect only users with certain countries, browsers, referrers, if even if they have a certain cookie. It also has WP CLI enabled for automating the entire thing!

Ghost

You’ll first need to go into your admin panel to Settings then click on Labs. Under Redirects click the “Download current redirects” link and you should get a file named redirects.yaml. It has two types of redirects and they work as list headers, 301 and 302. Once you’ve edited the file all you have to do is go back to where you downloaded it and upload your new one. Removing the redirects is as simple as uploading an empty .redirects.yaml file.

Scenario 1: Updating a single URL on the same domain

301:
  /old-postname/: /new-postame/

Scenario 2: Redirecting to a new domain

301:
  domain.com/post-slug/: domain22.com/post-slug/

Scenario 3: Using regex to redirect multiple pages while preserving the post-slug. So instead of writing each one out like this:

301:
  /article/article-1: /article-1
  /article/article-2: /article-2
  /article/article-3: /article-3

You could just have this one regex line and it would cover them all.

301:
  ^\/article\/([a-z0-9-]+)\/$: /$1

So it looks for the article directory and matches with whatever follows. The $1 is the article-1.

Hugo

So built-in to the core of hugo is a page method called Aliases . To be completely honest I don’t think it was meant for any of the use case scenarios we’ve been coming up with as solutions for in this post. Luckily, as with everything in Hugo, you can override it by creating your own version in your layouts folder. So this was taken from [Nick Herrig] (https://nickherrig.com/posts/hugo-external-redirects/ ) who did the leg work for us on this one.

So this is the original alias template that Hugo uses. We’re not going to use this but I wanted you to see how it uses the .Permalink and url variables.

<!DOCTYPE html>
<html lang="{{ site.Language.LanguageCode }}">
  <head>
    <title>{{ .Permalink }}</title>
    <link rel="canonical" href="{{ .Permalink }}">
    <meta name="robots" content="noindex">
    <meta charset="utf-8">
    <meta http-equiv="refresh" content="0; url={{ .Permalink }}">
  </head>
</html>

So what we are going to do is create our own alias template that’s better.
Put this in here: layouts/external-redirect/single.html

{{- template "_internal/alias.html" (dict "Permalink" .Params.url) -}}

Now whenever you want something redirected just put a file there with this frontmatter in it. Replace the URL parameter for where you’d like it to send the user.

---
type: "external-redirect"
url: "https://blog.roboflow.com/security-camera-monitoring/"
title: "Making Megadesk Smart"
date: "2024-03-17"
---

Easy 301 Redirects For IPFS Hosts (Cloudflare, Netlify, etc..)

So most IPFS hosts have an easy method to setup 301 redirects by putting a single file named _redirects in your root folder. This easy method has been fully embraced by Netlify and many other, lesser known hosts, however depending on your cache settings you might run into problems on Cloudflare (you should be using the cloudflare redirect rule anyways). So if you’re on Netlify or a host that offers ipfs hosting, you probably can just use this method.

/old-url /new-url 301
/another-old-url /another-new-url 301
/blog/aboutmycats.html  https://mycatsaremylifeandyouknowit.com

Really Easy 301 Redirects With Cloudflare

So I run this Hugo blog on Cloudflare pages. So far it’s been great, although it did take some messing around with the cache system to get it working the way I wanted it to. The beauty of using Cloudflare pages is that it comes with a lot of free utilities like a whole bunch of redirect options. The absolute easiest way to setup a 301 redirect in Cloudflare is with a Page Rule. You’re only allowed 3 in the free plan so if you use those up and don’t want to pay for more just switch to the Redirect Rules. If you have a whole lot of complicated rules you need to have them redirect there’s an option for Bulk Redirect Rules that is also covered under the free tier.

To set up a 301 Redirect via Cloudflare, log-in and select the domain involved. You should see towards the bottom of the left menu a category named Rules. Click on that and then on Page Rules. The incoming URI is the first option, in my case I only want to forward one specific URL so it’s pretty simple.

And there you go!

Buying Expired Domains To Redirect For SEO

I will keep this part short because there are countless resources already out that do a much better job that I could at explaining the actual strategy behind which domains to be looking at, but this topic is one that’s been argued about in the SEO community for 15+ years. Even employees at Google have made statements regarding link equity to known purchased domains won’t work only for the truth to come out later in a leaked company document . Here’s the truth, if you’re patient, know what to look for and know what you’re doing then this method absolutely still works. But most people only understand half of what it really means to find a good match. Or they think they have one but never looked at their dns records or what the pages looked like/ran on over the last year. Just using a redirect on any old domain you buy probably won’t do much for your own ranking. It has to be a clean domain that you can match up with your new content. Google keeps track of everything. From the number of recently added links to the history of your site AND the one that’s now redirecting to yours. If the content of the link doesn’t match up or if the anchor tag doesn’t describe the page you now have it going to it then Google won’t give you anything for those links. If that domain was recently used for malware or phishing before you purchased it then Google might actually punish you as you’re now linked with a toxic source.

Being Smart About Your 301 Redirects

I’ll say it again, do your research. One of those acts of due diligence is to always go and get all known links from the WayBack machine to investigate what the actual content looked like, the structure of the site and the tone of it all. Check out the source code on a random page, look at the sitemap if they have one in the archive, etc.. That’s just one out of more than a dozen or so points of interest you should be researching. If you’re still interested start heading down that google rabbit hole for more info. GoDaddy is one of the leading auction houses along with a few other registrars. Well established aggregators like ExpiredDomains.net can provide all of the domain info as well as more filters than you’ll know what to do with (in fact, how to use those filters should be one of your first searches). If you’re having a hard time finding the right domain research resources, here’s one that links just about every useful domain related site that I can think of.

Hope this helped you get those links moving!
~Brent

• The data is pulled from a script that checks last.fm every hour for updates
• The script is triggered by the workflow and if there’s been an update it rebuilds this page.
• The only services hooked up to send last.fm data are Spotify and Deezer
• That means you’re missing out on all my vinyl and anything streamed from youtube or archive.org!
• It was just a quick little hack I tried when I was new to Hugo so it’s basic.
• If you wanted to see more just check-out my Last.fm User Page
• Want to add one of these to your own hugo site? Add A Recently Played Music Page to Hugo

	Gonesville Artist: Bobby Weir Album: Blue Mountain Date Listened To: 16 Jul 2026
	A Whiter Shade Of Pale Artist: Johnny Blue Skies Album: Mutiny After Midnight (Extended Edition) Date Listened To: 16 Jul 2026
	Ain't That A Bitch Artist: Johnny Blue Skies Album: Mutiny After Midnight (Extended Edition) Date Listened To: 16 Jul 2026
	Everyone Is Welcome Artist: Johnny Blue Skies Album: Mutiny After Midnight (Extended Edition) Date Listened To: 16 Jul 2026
	Party All The Time Artist: Johnny Blue Skies Album: Mutiny After Midnight (Extended Edition) Date Listened To: 16 Jul 2026
	Dreadful Wind and Rain Artist: Jerry Garcia Album: Shady Grove Date Listened To: 16 Jul 2026
	Backwards Down the Number Line Artist: Phish Album: Joy Date Listened To: 16 Jul 2026
	Set Your Soul Free (Live, 11/3/2017) Artist: Trey Anastasio Album: TAB at the Fox Theater (Live) Date Listened To: 16 Jul 2026
	Althea - Live at Hartford Civic Center, October 14, 1983 Artist: Grateful Dead Album: Dick's Picks Vol. 6: Hartford Civic Center, Hartford, CT 10/14/83 (Live) Date Listened To: 16 Jul 2026
	Atlantic City > - Live 2019-11-25 Artist: Joe Russo's Almost Dead Album: Brooklyn, NY :: 2019-11-25 Date Listened To: 16 Jul 2026
	Fire On The Mountain Artist: Daniel Donato Album: A Young Man's Country Date Listened To: 16 Jul 2026
	Black-Throated Wind Artist: Bobby Weir Album: Ace Date Listened To: 16 Jul 2026
	Whiskey In the Jar Artist: Jerry Garcia Album: Shady Grove Date Listened To: 16 Jul 2026
	Brown-Eyed Woman - Live 2018-10-25 Artist: Joe Russo's Almost Dead Album: Live Oak, FL :: 2018-10-25 Date Listened To: 16 Jul 2026
	New Speedway Boogie - 2013 Remaster Artist: Grateful Dead Album: Workingman's Dead Date Listened To: 16 Jul 2026

How do you track what you listen to? What is Scrobbling?

Scrobbling is the term for sharing what content you are watching/listening to across other services. There are a bunch of sites like Last.fm which can directly interact with your media apps like Youtube, Spotify and Itunes to record what you are playing. The purpose is to get a better idea of what type of music you have enjoyed over time as well as help introduce you to new music based on your listening habits. You can enable this feature in the settings or options area of most media apps. While there are a bunch of sites that you can use to keep track of this data, I chose last.fm because of how easy it is to use (and free). So you will need a last.fm account for this as well as an API key.

Create a last.fm account and get an API key

Head on over to Last.fm and create an account. You will then need to head over to here to create the API key. Just put anything in for the application description. You can even use localhost as the callback url. The next screen will show you your application API key. Write this down somewhere safe.

Enable scrobbling in your media apps

This is just going to be for spotify but you can do the same for any other media application that you prefer. Log into your last.fm account, go to Settings then click Applications. Click on the connect to Spotify button and it will have you verify your credentials with spotify and that you are allowing last.fm to connect to your account. Once approved last.fm will keep track of what you listen to!

You Will Now Need To Add A New Shortcode

So in order for the list of recently listened to music to show up properly you’ll need to show a little bit of HTML code on the page. Hugo typically will remove any HTML tags on any pages unless you add a shortcode for displaying raw HTML. To do this create new shortcode template in layouts/shortcode/rawhtml.html

Then place the following inside it:

layouts/shortcode/rawhtml.html

<!-- raw html -->
{{.Inner}}

Now on your markdown page you’ll be able to use it like this;

{{< rawhtml >}} HTML GOES HERE {{< /rawhtml >}}

Here is the python script to update the page

You will need to go into your github repo, click settings then actions and add two new secrets - LASTFM_API_KEY and LASTFM_USERNAME with the appropriate info.

generate_markdown.py

import requests
import xml.etree.ElementTree as ET
from datetime import datetime
import os
import sys
import json

# Load track data from JSON file instead of fetching it directly from API.
try:
    with open("lastfm_tracks.json", "r") as f:
         response = json.load(f)

except FileNotFoundError as e:
    print(f"Error loading track data file {e}")
    sys.exit(1)  # Exit the script with a non-zero status to indicate failure.

# Extract track information from loaded JSON data.
tracks = []
for track in response['recenttracks']['track']:
    artist = track['artist']['#text'] if 'artist' in track else ""
    name = track['name'] if 'name' in track else ""
    album = track['album']['#text'] if 'album' in track else ""
    url = track['url'] if 'url' in track else ""

    # Correctly extract large image URL
    image_url = next((img['#text'] for img in track.get('image', []) if img.get('size') == 'large'), "")

    date_listened_full = track['date']['#text'] if 'date' in track else ""

    date_listened_date_only = date_listened_full.split(',')[0] if date_listened_full else ""

    tracks.append({
         'artist': artist,
         'name': name,
         'album': album,
         'url': url,
         'image_url': image_url,
         'date_listened': date_listened_date_only,
     })


# Get current date and time in ISO 8601 format with timezone offset
current_datetime = datetime.now().astimezone().isoformat()

# Generate Markdown content with Hugo front matter and embedded HTML for styling
markdown_content=f'''---
title: "Music"
date: "{current_datetime}"
#YOUR FRONTPLATE INFO GOES HERE
#I included how it changes the date to the current one every time it runs with {current_datetime}
#If you prefer to use the lastupdated option instead go for it
---

## Last Ten Tracks Listened To From Last.fm

{{{{< rawhtml >}}}}

<ul class="track-list">
'''

for t in tracks :
   markdown_content+= f'''
<li class="track-item">
<div class="track-image"><img src="{t['image_url']}" alt="{t['name']}"></div>
<div class="track-details">
<h3><a href="{t['url']}" target="_blank">{t['name']}</a></h3>
<p><strong>Artist:</strong> {t['artist']}<br>
<strong>Album:</strong> {t['album']}<br>
<strong>Date Listened To:</strong> {t['date_listened']}</p>
</div>
</li>'''

markdown_content += '''
</ul >
{{< /rawhtml >}}
'''

# Write to a markdown file- Change to whatever folder or filename you want the results placed in. 
with open("content/music.md", "w") as f:
     f.write(markdown_content)

Now what this does is it uses the lastfm_tracks.json file that your workflow will create from the last.fm API and generates a music.md file in your content folder. It currently is set to show the last 15 tracks but if you’d like to change that just scroll down and in the workflow change the appropriate line. You will also want to add all your frontplate information where noted as well as double-check where you’d like the generated file to go. When pushed live this page would be at yourwebsite.com/music

Now to the CSS to make it look decent

Depending on what theme you are using you are going to have to find what css file works for the type of page or post you are placing this in. For me that was post-single.css. All you have to do is copy that css file from your theme folder to your own assets/css/common/ folder and at the end add the following:

.css

.track-item {
    border-radius :8px; box-shadow :0 2px 4px rgba(0 ,0 ,0 ,.1); margin-bottom :20px; overflow:hidden; display:flex; align-items:center;
}

.track-image {
width:auto;height:auto;margin-right:.5em;
}

.track-image img{
max-width :100%;height:auto;border-radius:.5em;
}

.track-details{
padding:.5em;width:auto;
}

.track-details h3{
margin-top :0;margin-bottom :.25em;font-size :
1.25em;} p{margin :.25em;font-size :.9em;color:d3d3d3;}

Now to keep this updated!

In order to keep this regularly updated we need to create a github workflow. What this will do is grab the latest list of songs recorded by last.fm’s API and save it as a json file named lastfm_tracks.json. That’s what our python script will use to fill out the music.md page as well as what the workflow will check against to see if there has been an update to our recently listened to list. More information on workflows can be found here . In your base directory create two new folders .github and then inside that create a folder named workflows (so /.github/workflows/) In the workflows folder we are going to create a file named update_music_page.yml with the following code:

update_music_page.yml

name: Update Last 15 Tracks

on:
  schedule:
    - cron: '0 */4 * * *' # Runs every 4 hours (adjust as needed)
  push:
    branches:
      - main # or your default branch

jobs:
  update-tracks:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11.2'

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install requests jq          
      - name: Check for new tracks on Last.fm
        id: check_tracks
        env:
          LASTFM_API_KEY: ${{ secrets.LASTFM_API_KEY }}
          LASTFM_USERNAME: ${{ secrets.LASTFM_USERNAME }}
        run: |
          # Load last fetched tracks from file or set default value if not exists
          if [ -f ./lastfm_tracks.json ]; then  # Look in the current directory (root)
            LAST_TRACKS=$(cat ./lastfm_tracks.json)
            LAST_TIMESTAMP=$(echo "$LAST_TRACKS" | jq '.recenttracks.track[-1].date["uts"]' | tr -d '"')
          else
            LAST_TRACKS="{}"
            LAST_TIMESTAMP=0 # Default value if no previous data exists.
          fi

          # Fetch recent tracks from Last.fm API and filter by timestamp. Change # after limit if you want something other than the latest 15 tracks. 
          RESPONSE=$(curl -s "http://ws.audioscrobbler.com/2.0/?method=user.getrecenttracks&user=${LASTFM_USERNAME}&limit=15&api_key=${LASTFM_API_KEY}&format=json")

          # Print response for debugging purposes
          echo "API Response: $RESPONSE"

          # Check for valid response before processing
          if [[ "$RESPONSE" == *"error"* ]]; then
            echo "Error fetching data from Last.fm API."
            exit 1  # Exit with error code.
          fi

          NEW_TRACKS=$(echo "$RESPONSE" | jq --arg LAST_TIMESTAMP "$LAST_TIMESTAMP" '.recenttracks.track | map(select(.date.uts != null and (.date.uts | tonumber > ($LAST_TIMESTAMP | tonumber))))')

          echo "New Tracks Data: $NEW_TRACKS"

          NEW_TRACKS_FOUND=false

          if [ "$(echo "$NEW_TRACKS" | wc -l)" -gt 0 ]; then 
            echo "New tracks found."
            NEW_TRACKS_FOUND=true
            
            # Save the latest track information to a JSON file in root directory.
            TRACK_DATA=$(echo "$RESPONSE" | jq '.recenttracks.track')
            
            echo "Writing new track data to lastfm_tracks.json..."
            
            echo "{\"recenttracks\":{\"track\":$TRACK_DATA}}" > ./lastfm_tracks.json
            
            if [ $? -eq 0 ]; then
              echo "Successfully wrote to lastfm_tracks.json."
              # Get latest timestamp from new tracks for next run comparison.
              LATEST_TIMESTAMP=$(echo "$NEW_TRACKS" | jq '.[-1].date.uts // empty' | tr -d '"')
              echo "Latest timestamp updated to $LATEST_TIMESTAMP."
              
              # Set output as environment variable
              echo "has_new_tracks=true" >> $GITHUB_ENV

              # Check for changes before committing
              git config --global user.name "${{ github.actor }}"
              git config --global user.email "${{ github.actor }}@users.noreply.github.com"
              
              git add ./lastfm_tracks.json 
              
              # Only commit if there are changes in lastfm_tracks.json.
              git diff-index --quiet HEAD || git commit -m "Update lastfm_tracks.json with new tracks"
              
           else 
             echo "Failed to write to lastfm_tracks.json."
             exit 1  # Exit with error code on failure.
           fi
            
          else
            echo "No new tracks found."
            # Set output as environment variable
            echo "has_new_tracks=false" >> $GITHUB_ENV
          fi          


      # Check for changes in git files (including pushes)
      - name: Check for new commits or changes in lastfm_tracks.json
        id: check_git_changes
        run: |
          git fetch origin main 
          
          # Check if there are any changes between HEAD and origin/main (the last commit)
          if ! git diff --quiet HEAD origin/main; then
            echo "has_new_git_changes=true" >> $GITHUB_ENV
          else
            echo "has_new_git_changes=false" >> $GITHUB_ENV
          fi
                          
      # Conditional execution based on whether new tracks were found or there are new commits/pushes.
      - name: Run script to generate markdown file
        if: env.has_new_tracks == 'true' || env.has_new_git_changes == 'true'
        env:
          LASTFM_API_KEY: ${{ secrets.LASTFM_API_KEY }}
          LASTFM_USERNAME: ${{ secrets.LASTFM_USERNAME }}
        run: |
           python generate_music_markdown.py           

      - name: Commit changes
        if: env.has_new_tracks == 'true' || env.has_new_git_changes == 'true'
        run : |
           git config --global user.name "${{ github.actor }}"
           git config --global user.email "${{ github.actor }}@users.noreply.github.com"
           
           git add content/music.md lastfm_tracks.json 
           
           # Only commit if there are changes in music.md or timestamps.
           git diff-index --quiet HEAD || git commit -m "Update music.md and track list"           

      - name: Push changes back to repository
        if: env.has_new_tracks == 'true' || env.has_new_git_changes == 'true'
        run: |
           git config --global user.name "${{ github.actor }}"
           git config --global user.email "${{ github.actor }}@users.noreply.github.com"

           # Push changes to main branch
           git checkout main
           git push origin main           
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}EN }}
	  branch : main

This is set to run every 4 hours but you can change that to however frequent you would prefer by changing the line under cron. You should also change the folder and file that it pushes to your git repository unless you want to keep it at contents/music.md.

Git add, commit and push to your repo!

Hopefully there won’t be any errors along the way and it should run the python script, create your new music page and stay dormant until it is ready to update again. What this will not do is auto-push to your hosting service unless your repo is already setup with either another workflow or service like cloudflare where it will automatically update anytime there is a new build. Pushing it to your host of choice though only needs adding another job entry in your yml file though so it should be simple enough. If there are errors you can see them under Actions in your github repo. I currently have it giving updates as it goes through each process so finding where the problem is shouldn’t be too hard.

Hopefully this helps some of you, please use it and create something much cooler as this is pretty basic but let me know what you have made so I can check it out!

Enjoy!

~Brent

The only problem was Hugo will naturally ignore any HTML in posts/pages unless you do the following:

Add a shortcode to your site

It’s simple enough, just add a shortcode template in layouts/shortcodes/rawhtml.html with the following content:

<!-- raw html -->
{{.Inner}}

Then in whatever post or page you’d like to use html in just enclose it in these tags:

{{< rawhtml >}}
<p><h2> Brentter.com has all the <strong>BEST</strong> hugo tips</h2></p>
{{< /rawhtml >}}

And the output will follow your HTML tags:

Brentter.com has all the BEST hugo tips!

Took me a hot minute to figure that out so hopefully this speeds up your discovery process a bit.

Enjoy!

~brent

Follow is the latest app to try and tackle the world of RSS aggregation and I think they might have done it.
It is being developed by the same group behind RSSHub , the folks who created a unique way of being able to create RSS feeds for websites and platforms that typically didn’t have them (or in some cases, didn’t want them). It’s currently in alpha-phase so an invitation is necessary to join but I was lucky enough to get one early. It’s currently available for Linux, OSx, Windows and there’s a web-version. My current solution for RSS aggregation is a self-hosted version of FreshRSS. It’s great as far as being easy to setup and light on resources but it’s not so easy to get through your unread content to find something you want if it’s been a while since you’ve gone through your feeds (as in, missed a day or two). It’s an RSS feed reader like all the others, nothing special other than how little server resources it requires for an insane amount of feeds. So I’m always looking out for something new, and here’s why I think Follow has potential.

Design and Layout

The feeds are designated by a global category of Notifications, Audio, Video, Pictures, Social Media and Articles. The layout for each is somewhat similar but I’ll show you the difference and why it works well. In the Settings area you have a wide variety of options from selecting your own choice of font for not only the UI of the app but also for the content.

It has the option to render the inline style of the original web page (which is a feature I haven’t seen before for a RSS reader) and a wide selection of options for what style to render any code in (I tend to stick with nord). Built-in integration with Eagle, Readwise and Instapaper is also provided if you’re migrating over from those sites. Speaking of which, if you already are using a RSS reader you can import OPML files so moving your collection over is a one click process. Seeing how some of the main developers speak Chinese, the app has built-in support for multiple languages as well as the option to have content spoken out loud to you in a variety of different style of voices. Now for the content itself.

Photos

First off let’s go with one of the sites I am following that is photo-centric, The NASA Picture Of The Day site. It arranges the photos nicely, loads them quickly and if I click on one it will enlarge it in-screen so I don’t have to load the whole page. If I wanted to visit the actual page in my browser I’d just click on the title/website name below the image. It makes browsing image sites a breeze.

Videos

Here’s just a sampling of some of the youtubers I tend to enjoy, as you can see this my HomeLab folder and these are the latest from the grouping (all homelab rockstars in my opinion). If you mouse over any of the videos it will start to play them muted so you can see the basic content and if you click on one of them it will quickly load & start the video full screen in the app. The video can also be manipulated using keyboard shortcuts so you’re not constantly having to click on the controls. It’s a much more of an enjoyable experience than on the actual Youtube website but if you prefer that then you can also just click the video title/channel name and it will open in your browser. I was impressed with just how fast each video loaded, even from sites that were hosting their videos themselves.

Articles

So this is usually the meat and potatoes of most RSS readers, a steady collection of blogs/websites separated into category or tags. There’s no difference here, you can setup as many folders as your heart desires and check out the latest from a single site, all the sites in a folder or all the sites together. You can easily mark any grouping as read/unread and even reload older content if you want to go back to something from the past that you already looked at. If one of your sites doesn’t naturally allow all the content in the rss feed you can click the Readability button in the top right of the app and it will attempt to load the content. So far the only problem I’ve come across using that feature was on a few sites where only the text of the post would load.

Audio

So when I’m out at the gym or walking around town I tend to listen to podcasts and Follow has its own built-in audio player. In the image above you can see an example of one of my favorite podcasts to listen to, Darknet Diaries . The text from the episode page has loaded on the right but if you look in the bottom left corner you can see that the podcast has started to play in it’s own little audio player. If you mouse-over the player you have options to speed the audio up, go back or forward 10seconds, visit the podcast site and even an option to download it. Just like with the video player, these can all be controlled via shortcuts.

Notifications

So this is one of my favorite parts of this app. Using the RSSHub integration, you can create feeds for a lot of important content that usually would be difficult to run in an RSS reader. The example on screen is a feed that tracks DIYGod ’s (one of the main developers of Follow) github activity. As you can see this was a git push to one of his repositories and includes his commit message and the accompanying tag associated with it (which if you click on will take you to the github page).

While that was a last minute addition for this post, I do have a folder following github repo issues for a few projects that makes it easy to track. In addition to that I have a “what’s trending’ folder that has feeds for what’s currently hot on producthunt.com, github trends for python and go, the latest in HackerNews:Show and a feed for Google’s Daily Search Trends US. You can customize it even further if you wanted to (think: keyword or brand tracking). It makes keeping up with what’s fresh very easy.

Social Media

So just like the category says, this is where I list some feeds from social media. Some of these accounts I actually follow and just don’t want to have to log in to see whats new and some are just fun to read. It’s a mixture of twitter and mastodon and the app tracks them perfectly. It even shows the embedded emojis correctly. Something that you can’t see very well in this photo but I will explain how it works next is that you’ll notice all of the social feeds on the screen are in English. On twitter these were written in Chinese. How did it get translated? That brings me to the next cool feature, Actions.

Actions

So Follow has a feature called Actions where you can setup little automatons within the app itself. The one pictured is my automation where if the content is from the social media section, it auto-translates it to English. Very helpful when you’re following a multi-cultural crowd of developers. They’re working on adding more features to it but for now you can use it to filter literally anything and then perform actions ranging from having the content summarized by AI, translated, blocked or even rewritten. It has the ability to also add even more actions depending on triggers of that first one. This looks like it’d be great for brand management. You can set it to track mentions of your brand or @’s on twitter and then have it perform an action as a result.

Adding Feeds

So here’s how you add a new feed in Follow. You can either just type the website in the search box and it will go off and try to auto-discover the feed (which sometimes works, sometimes doesn’t) or you can paste the actual URL to the feed itself. Either way it will bring up a window showing all of the feeds that have been found with a little preview asking if you want to add it and what folder would you like to put it under (also you can change the name of the feed). Where this gets interesting is the integration with web3 and RSSHub. So this app allows you to follow someone’s crypto account if they’re based on certain platforms (i.e. RSS3). You can track their socials, trades/transactions, collections, nfts, etc.. You can also add feeds using the RSSHub integration which is how I was easily able to add specific github and DockerHub feeds. They have a whole slew of different options and app feeds available. You can find that list on The RSSHub Routes page. They even have an integration for pornhub feeds, if that’s what you are into.

RSS3/Crypto Framework

So this entire app is linked with a web3 blockchain that grants each user 20 free $POWER crypto each day as long as you hit the “claim” button. You can use these to tip feeds that you like (and the owner can claim that tip), you can use them to claim ownership of your own feed/content, and currently you can use them to generate invite codes for Follow (it requires 100 coins per invite so it’ll take 5 days for most users to get their first one). I’m not too keen on crypto but in the screenshot you can see my current balance (and that I recently bought an invite for someone) as well as a little more information on the service. It’s all on the VSL blockchain.

The AI Top News Of Today/Yesterday/Night Before Last

Here’s another feature I’ve never seen used before. For any category you can read what the AI thinks the top stories (from your feeds) are of today, yesterday or the day before that. It might need a little work but it’s a great feature if you just want to see the highlights of what’s going on.

Public Profile

The last thing I’ll show you from the app is that everyone gets their own public profile page. You can decide what doesn’t show up on it by making a feed “private” but it’s basically a list of who you follow. Currently you don’t get a personalized URL, but that is a feature that they claim is coming soon. This is another great branding idea once they implement that as you can have feeds related to your own work that go beyond just your blog url (product updates, social media feeds of users saying positive things about you, etc..). Here is a link to mine if you want to see what it looks like (I’ve only just started using this so there’s still a whole lot more sites I still need to add). Brentter’s Profile

So there you have it. I’m obviously a fan, and seeing how it’s a free app, I’m definitely not a paid one either. While there are still bugs popping up here and there it seems like they’re getting worked on pretty quickly by the looks of their very busy github page. Join their discord if you’re looking for an invite (though I’d also follow their developers on twitter too as they tend to post invite codes in both places).

Cheers,
Brent

So let’s start with the basics.

To download a single file:

wget http://example.com/supercool.zip

Multiple files? Just add a space between them

wget https://example.com/supercool.zip https://example.com/superbad.zip

If you have a whole list of files you need to download throw them in a txt file.

wget -i urls.txt

Use -O if you want to rename the file you’re downloading (it’ll save the file as homework.zip):

wget -O homework.zip https://example.com/xxx.zip

If you’re getting rate-limited or have a bad connection, you can have it keep retrying (50 times in this case):

wget -tries=50 https://example.com/cool.zip

Did you spill your chocolate milk on the keyboard mid-download and it aborted? No problem, resume downloads with -c

wget -c https://example.com/10000000gig.zip

Change the user agent and pretend to be a web browser

✅ Here’s a list of updated user agents to use ✅

wget --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0" https://example.com/cia.zip

Ok, now let’s say you have a directory full of links you want

wget -r -nH --cut-dirs=2 -nr -R "index.html*" http://website.com/dir1/dir2/data

• The -r tells it to download recursively
• The -nH tells it not to include the domain name.
• We don’t need those first two directories so –cut-dirs=2 ignores them
• The -nr makes sure it doesn’t go back a directory if there’s a backlink somewhere on the page
• The -R “index.html*” rejects downloading any index.html files because this is a link directory.
  That means all you will download are the links, and they won’t be in two levels of empty directories with blank html files and long URLS.

Now let’s download the entire website

wget --mirror -page-requisites -k -P TheFilesGo/Here/ https://www.funtimesbiz.com

• The -mirror tells it to download everything structure wise identical to the site’s original content
• The -page-requisites tells it to also grab any secondary files like css, js and images
• The -k tells it to convert any links to the local directory structure so they’re not broken in your local copy
• The -P TheFilesGo/Here tells it to download everything into that directory instead of the local one.

Now, let’s say you’re grabbing a lot of files and you don’t want to get rate-limited/banned, you can add -w 3 and it will wait three seconds between transfers (adjust as needed). If you also added after that --random-wait it would randomize the time between each transfer between 0.5 and 1.5 seconds of the value chosen with -w.

A quick run down of other useful Wget options:

• -q is for quiet mode, only major notifications will be shown.
• -A gif,jpg would tell it to only download gif and jpg files.
• -A "resume" would only download files that start with the word resume.
• -R "cupcakes" would NOT download anything with the word cupcakes.
• -o logfile.log will put all the info it would have posted to the screen in a logfile.
• -b will download files in the background, useful when you have other stuff to do in the terminal.
• --no-check-certificate will skip any certificate check in case the source has an invalid or missing SSL cert.
• --load-cookies cookies.txt will load the cookies from the text file and send them to the site, great for bypassing having to re-login each time.
• --save-cookies file.txt will save any non-expired cookies to file before exiting the task.

Now you should be ready to scrape that site you’ve been after. A fun site to practice on is The-Eye.eu as it contains terabytes of directory links/media.

Wget is your friend. DNS is your enemy.

Computers

• MacBookPro 11,1 (2014?)

  • 8GB DDR3
  • Intel Core i5-4258U x 2
  • Mesa Intel Iris Graphics 5100 (HSW GT3)
  • 250gb HD
  • Debian 12 Bookworm w/ Xfce DE (with the Plank dock becuase I love it)

• My Gaming/Workstation PC

  • ASUS Prime Z590-P Motherboard
  • 11th Gen Inten Core i7-11700K @ 3.6GHz
  • NVIDIA GeForce RTX 3080Ti
  • 64GB ddr4 (2x 32GB)
  • 2TB NVMe
  • 27" Asus ROG PG278Q (2560x1400 @ 144hz) - for play
  • 24" Dell S2421NX (1920X1080 @ 60hz) - for work.
  • Windows 10 Pro

Applications

• Terminal w/ oh-my-zsh running gruvbox theme for linux + WSL
• Terminal w/ oh-my-posh running gruvbox theme & Terminal Icons for Powershell
• Alacritty with Oh my Zsh and Tmux for the laptop.
• Spotify w/ Spicetify mod - It makes the app so much better.
• Steam - Gaming
• Discord - Gaming and keeping in touch with friends/community
• EarTrumpet - Best windows sound manager ever and it’s free.
• Git CLI - local Gitea for private stuff, github for everything else
• VSCode w/ you guessed it, the gruvbox theme.
• Obsidian - I use this religiously, with templates and plugins you can do literally anything with this free software.
• Raindrop.io - One of the best bookmark apps around…free as well.
• Follow - RSS Feed Reader
• Firefox w/ uBlock Origin, Bypass Paywalls Clean , Multi-Account-Containers , Raindrop.io and 1password.
• ShareX - Best screenshot app out there (and it’s free).
• Flow Launcher - Great keyboard shortcut/launcher/task-master for windows (also free)
• Qbittorent - for linux isos of course.
• Figma & Affinity Designer 2 for design/mockups
• Gimp for photoshop needs
• ProtonVPN - For when I’m out and about
• ProtonMail - recently switched off of my google workspace plan and have never been happier.
• 1Password - Been using them since they first came out and they’ve only gotten better.
• Docker

Mobile

Iphone 15 Pro

• ProtonMail
• 1Password
• Firefox
• Authy - I regret having used this as it’s one of the few apps you can’t export your accounts from in case you want to move to a different provider.
• Carrot - It’s a weather app with an attitude, also pretty damn accurate.
• Signal - End to end secure communications
• Ice Cubes - Mastodon client
• NetNewWire - To connect to my FreshRSS setup

Sound/Music

• Sennheiser 599 Open back
• Linsoul Kiwi Ears Quintet IEM
  - SMSL D10 Pocket Portable Headphone Amp (this and the Linsoul Kiwi are my everyday carry)
• Spotify, SomaFM , ReListen for all the grateful dead on archive.org and then my own instance of Navidrome

Homelab

• EliteDesk 800 G2 Mini - 1TB SSD, 500GB NVMe, 32GB DDR4 Ram

  • Running Proxmox which hosts:
    • Navidrome - Lightweight multi-user music streaming to any device
    • Radarr, Sonarr, NzbGet, ruTorrent, Overseerr, Plex - Media Aquisition
    • ChangeDetection.io - Monitor any webpages to see if any part of them change and get notified.
    • ActivePieces - Free open-source Zapier replacement
    • FreshRSS - So many feeds!
    • Homarr - Customizable homepage for easy access & stats on all my self-hosted docker containers
    • Gitea - local git (But I want to try Soft-Serve )

• Raspberry Pi 4

  • Pi-hole - network ad blocking & DNS server.

Hosting

• Porkbun - I use to own a LOT of domains and have used many registrars but Porkbun right now is the best in my opinion, great customer support, they’re US based (Oregon!), a working API and constant deals. Plus they haven’t had any of the issues that other big registrars have faced regarding locking people out of their accounts, getting social-engineered or straight up hacked. Plus they’ve made little piggie icons for their popular domains.
• ZeroSSL - All the same benefits of Let’s Encrypt but with no rate limit, a working API and you can verify via email
• Hetzner - Not only is their cloud offering on decent equipment, but fast and cheap. Why rely on 3rd party services when you do it yourself?
• IOFlood - One of the best bare-metal dedicated server hosts in the US. They only do one thing, bare-metal servers. And they’re a no bs, here’s the testing/Passmark scores on this rig, honest type of guys. Plus they have servers in various configs. 10Gbps connections out of Phoenix, AZ.
• Cloudflare - CDN/DDOS protection on the cheap (free depending on your needs)
• BackBlaze - $6 per TB and they free egress if downloading through one of their partners like Cloudflare.
• Fastmail.com and Protonmail - Both cheap, secure and have great support for teams/small biz. I’ve used both.

I’m skipping a bunch of categories and most likely leaving a bunch of stuff out but if you have any questions, especially about self-hosting or home-networking in general feel free to reach out! I love talking about FOSS and DIY setups.

Cheers,

~Brent

Ever wanted to see everything the WayBack Machine has saved for a domain? Ever wanted to get all of it at once? Here are a couple tools including an entire WayBackMachine OSINT framework giving you everything you need to get everything you want from the archive. Here’s the easy one, waybackurls.py:

Just give it the url you want it to search for and if you DO want it to include subdomains add True at the end.

python3 waybackurls.py example.com true <—with subdomains

python3 waybackurls.py example.com <—without subdomains

It’ll save the results in a nice little json file in the same directory.

import requests
import sys
import json


def waybackurls(host, with_subs):
    if with_subs:
        url = 'http://web.archive.org/cdx/search/cdx?url=*.%s/*&output=json&fl=original&collapse=urlkey' % host
    else:
        url = 'http://web.archive.org/cdx/search/cdx?url=%s/*&output=json&fl=original&collapse=urlkey' % host
    r = requests.get(url)
    results = r.json()
    return results[1:]


if __name__ == '__main__':
    argc = len(sys.argv)
    if argc < 2:
        print('Usage:\n\tpython3 waybackurls.py <url> <include_subdomains:optional>')
        sys.exit()

    host = sys.argv[1]
    with_subs = False
    if argc > 3:
        with_subs = True

    urls = waybackurls(host, with_subs)
    json_urls = json.dumps(urls)
    if urls:
        filename = '%s-waybackurls.json' % host
        with open(filename, 'w') as f:
            f.write(json_urls)
        print('[*] Saved results to %s' % filename)
    else:
        print('[-] Found nothing')

This is the very same wayback script you’ll find in Kali Linux and it originated from a github gist back in 2017 by mhmdiaa

Since then a bunch of variations have sprung up including one written in the go language .

Once you’re ready, there’s actually a whole python package/cli library that’s pretty useful in interacting with all 3 of the API systems that they have for the WayBack Machine. It’s called Waybackpy . It’s got classes built in to work with the SavePageNow/Save API, their CDX Server API and the Availability API. That one’s great because it has built in functions for only grabbing snapshots or the closest saves to a specific date or even the newest vs the oldest recorded archive for a page. You can download that one using pip or conda:

pip install waybackpy -U

conda install -c conda-forge waybackpy

Now that you’re pretty seasoned in using the WayBack Machine, I think it’s time for you to learn just how much information the Wayback Machine actually has inside those archives. It’s more than just html, javascript and images.. it’s a goldmine of data.

Chronos - An OSINT Framework For The Wayback Machine

And then, like a WayBack Machine demi-god looking down on us mortals and feeling a bit generous, mhmdiaa, the very same one who wrote the script featured above, gave us Chronos . It’s an entire OSINT framework for the Wayback Machine Archives. Written in go it offers not only a lot of fun oportunities but also a whole lot of options to modify the output/parameters to whatever your need might be.
A couple highlighted features:
- Extract endpoints and URLs from archived JavaScript code

    chronos -target "spltjs.com/*" -module jsluice -output js_endpoints.json

• Calculate archived favicon hashes

    chronos -target "playboy.com/favicon.ico" -module favicon -output favicon_hashes.json

• Extract archived page titles

    chronos -target "github.com" -module html -module-config "html.title=//title" -snapshot-interval y -output titles.json

• Extract paths from archived robots.txt files

    chronos -target "pornhub.com/robots.txt" -module regex -module-config 'regex.paths=/[^\s]+' -output robots_paths.json

• Extract URLs from archived sitemap.xml files

    chronos -target "bestbuy.com/sitemap.xml" -module xml -module-config "xml.urls=//urlset/url/loc" -limit -5 -output sitemap_urls.json

• Extract endpoints from archived API documentation

    chronos -target "https://docs.gitlab.com/ee/api/api_resources.html" -module html -module-config 'html.endpoint=//code' -output api_docs_endpoints.json

• Find S3 buckets in archived pages

    chronos -target "github.com" -module regex -module-config 'regex.s3=[a-zA-Z0-9-\.\_]+\.s3(?:-[-a-zA-Z0-9]+)?\.amazonaws\.com' -limit -snapshot-interval y -output s3_buckets.json

Modules include:

- regex - Extract regex matches
- jsluice - Extract URLs and endpoints from Javascript code using jsluice
- html - Query HTML documents using Xpath expressions
- xml - Query XML documents using XPath expressions
- favicon - Calculate favicon hashes 
- full - Get the full content of snapshots

Command-Line Options:

Usage of chronos:

• target string
  • Specify the target URL or domain (supports wildcards)
• list-modules
  • List available modules
• module string
  • Comma-separated list of modules to run
• module-config value
  • Module configuration in the format: module.key=value
• module-config-file string
  • Path to the module configuration file
• match-mime string
  • Comma-separated list of MIME types to match
• filter-mime string
  • Comma-separated list of MIME types to filter out
• match-status string
  • Comma-separated list of status codes to match (default “200”)
• filter-status string
  • Comma-separated list of status codes to filter out
• from string
  • Filter snapshots from a specific date (Format: yyyyMMddhhmmss)
• to string
  • Filter snapshots to a specific date (Format: yyyyMMddhhmmss)
• limit string
  • Limit the number of snapshots to process (use negative numbers for the newest N snapshots, positive numbers for the oldest N results) (default “-50”)
• snapshot-interval string
  • The interval for getting at most one snapshot (possible values: h, d, m, y)
• one-per-url
  • Fetch one snapshot only per URL
• threads int
  • Number of concurrent threads to use (default 10)
• output string
  • Path to the output file

The last blog post on brentter.com was on December 29, 2011. It had been a while since I’d left my old day job in the advertising industry and really didn’t want to write about it anymore. So instead, I wrote about something I had done earlier that day for one of my clients, Use SQL queries to bulk-modify thousands of wordpress pages. There’s no irony in that being the last post seeing how a few years before I had to figure out how to change the video embed tags from my previous video host (which went bankrupt and 2 days later deleted all their videos) to youtube. When that happened all of a sudden 99% of my pages had non-working video embeds. While some were backed up, there were a lot that only lived on that now dead video host. As I re-uploaded what I could over on youtube that meant I also had to change out all of the embed tags. Now I won’t lie, my first instinct was to be lazy about it. Back then I was using wordpress/mysql. I did what most wordpress users would do in that situation and I searched for a plugin to do it for me. But low and behold, wordpress and php back then just weren’t a good combination for a long-running, resource intense task like the one that I was in need of. This had to be dealt with more directly. Every post was stored in a mysql table for the blog so all I needed to do was have it search for instances of the video tag and swap it out for the new one. So there it was, the worst day of this site ended the same way the last day of this site would years later, with SQL. At least it wasn’t DNS.

So Why Did You Care So Much About The Site?

Believe it or not, this website once was popular. It was consistently ranked as one of the top 50 advertising blogs on the internet (though this was back when very few people actually had blogs) and I was making a decent amount of money every month through sponsors and site traffic. Getting the content was easy back then too, I had emailed just about every advertising, PR and production agency I could find proposing them this great offer: Send me your ad campaigns before you launch them and I’ll give you free publicity. To add to that for emails that I’d send to creative departments is I would also post whatever attribution/credits you send along with it along with the ad. For production companies I’d email their creative directors and/or copywriters. While a PR agency might have seen a blog as not worth the effort, the folks who work in creative jumped at the chance of their names being posted along with their work. Eventually that led to me being asked to write articles on other marketing sites, most of which are now also dead, and scored me free passes to all the AdTech conferences in SF, Chicago and Miami (Thanks to Steve Hall for that, they were truly great memories). I’d eventually leave traditional advertising and with that maintaining this advertising blog became more and more of a chore than a fun hobby so one day I just decided to stop.

Why The Change Of Heart?

Even though I wasn’t writing in a public forum anymore, my ADHD practically mandates that I write notes for myself otherwise I wouldn’t be able to get anything done in my life. While the apps & methods I use to write have changed over the years I still find myself jotting down ideas, bookmarking new finds and cultivating a never-ending stream of content that up until now I kept to myself. This will be a slow change. I’m in the process of consolidating all my stored data, including the multiple sets of RSS aggregators I’ve set up through the years to keep track of interesting content sources by topic/industry. The logs on those alone are an insane amount of data. Everything needs to be cleaned & organized. I’m still debating how I’m going to go about doing that and whether or not it’s even worth it for 99% of the old content. Either way, the architecture of this site will most definitely change over the next few months.

Some Final Thoughts

One thing I really enjoyed about writing all the time on here and other blogs was it gave me a chance to meet new people, even if only for a brief exchange of comments. Tuning in to strangers and seeing what they’ve made has led to some incredibly rewarding experiences including some that eventually migrated from being online friends to working on projects together & meeting in real life. While there’s always going to be the dark side of the internet and all the strangers who feel anonymous sending messages on it, over-all it was a positive experience. While the internet now is just example after example of enshittification , I think a return to the old ways of the web is just what we need.

Welcome to my digital garden, I hope you find something interesting while you are here.

~brent

This websites name is Everythings Better With Brentter. It’s been hosted here on brentter.com for over twenty years by me, Brent Terrazas. I bought brentter.com on November 19th, 2004 and have owned it ever since. Believe it or not this was actually the second domain (out of 100+ since) that I’d own. My day job back then was in advertising so it made sense for me to write about the industry as I’d been constantly writing online for years at that point. I loved it, and because of that I ended up writing A LOT on here and other advertising industry sites. This website would even become one of the top 20 ranked advertising blogs at one time but somewhere along the way my day job turned into two jobs and then creating content for this site felt like it was becoming a third. It eventually led to me deciding that I needed a change in my life and the best way to do that was to take a break from writing. With that in mind I did what any normal mid-twenty-something kid would do and I disabled comments site-wide then promptly signed up for an adult kickball league with some friends. I loved it.

6 months of no posts and I eventually gave it one last entry. It was about what I had just finished doing for a client. I had been writing up a bunch of SQL queries to bulk-edit a few thousand of his WordPress pages so that he wouldn’t have to go through and edit them one by one. Then about a year or so later I decided to just take it offline. The DB was actually still there, just I’d removed all the static files and replaced the homepage with some ASCII. This site was running WordPress and at that time the platform was just a security nightmare, especially if you weren’t actively monitoring it. I mean, it still is a security nightmare. If you’ve been keeping up with what’s been going on between the co-creator of Wordpress and WP Engine you’d also think the entire future of the platform seems like a nightmare too.

So You’re Going To Start Posting Advertising Commercials Again?

Well, maybe a few, but just some of my old favorites. With all the toxicity that now seems to be everywhere online I think we should all try to remember what it was like in the early days. Back when you were guaranteed at least a dozen AOL cds in the mail each week and AltaVista was the largest search engine. You’d go around and find sites that were there to share content with other people without looking for a marketing angle. You’d find their favorite blogrolls and web-rings, collections of stories or cool animated gifs - just sharing for the sake of sharing. I like that idea so I’m going to follow it.

How’s Hugo?

So far I’ve been enjoying working with Hugo. They really weren’t kidding when they talk about just how fast it is. The only downsides have been in the template system and how janky it can be at times and that every big update to Hugo means something important in the blog is going to break. It’s a work in progress.

Cheers,

~Brent
Everythings Better With Brentter

(Here’s a few spots where I can be found elsewhere on the web):

	Brent Terrazas Send Me Mail
GitHub	Last.FM	Gravatar
IndieWeb	Reddit	Mastodon
Ko-Fi	Follow.is App	Steam
Keybase.io	Flickr	Twitter/x
Libre.fm	Bsky.App

]]>