Ask HN: What overlooked class of tools should a self-taught programmer look into
360 nathanasmith 6 hrs 173
news.ycombinator.com/item?id=19900955
15 years ago I learned Python by studying some O'Reilly books and I have been a hobbyist programmer ever since.
The books went into detail and since reading them I've felt confident writing scripts I needed to scratch an itch. Over time, I grew comfortable believing I had a strong grasp of the practical details and anything I hadn't seen was likely either minor quibble, domain specific, or impractically theoretic.
This was until last year when I started working on a trading bot. I felt there should be two distinct parts to the bot, one script getting data then passing that data along to the other script for action. This seemed correct as later I might want multiple scripts serving both roles and passing data all around. Realizing the scripts would need to communicate over a network with minimal latency, I considered named pipes, Unix domain sockets, even writing files to /dev/shm but none of these solutions really fit.
Googling, I encountered something I hadn't heard of called a message queue. More specifically, the ZMQ messaging library. Seeing some examples I realized this was important. The step of then plowing through the docs was nothing short of revelatory. Every next chapter introduced another brilliant pattern. While grokking Pub/Sub, Req/Res, Push/Pull and the rest I couldn't help breaking away, staring in space, struck by how this new thing I had just read could have deftly solved some fiendish memorable problem I'd previously struggled against.
Later, I pondered the meaning of only now stumbling on something so powerful, so fundamental, so hidden in plain sight, as messaging middleware? What other great tools remain invisible to me for lack of even knowing what to look for?
My question: In the spirit of generally yet ridiculously useful things like messaging middleware, what non-obvious tools and classes of tools would you suggest a hobbyist investigate that they otherwise may never encounter?
aequitas 4 hrs
Makefiles. I always dismissed them as a C compiler thing. Something that could never be useful for Python programming. But nowadays every project I create has a Makefile to bind together all task involved on that project. From bootstrapping the dev environment, running checks/test, starting a devserver, building releases and container images. Makefiles are just such a nice place to put scripts for these common tasks compared to normal shell scripts. The functional approach and dependency resolving of Make allows you to express them with minimal boilerplate and you get tab completion for free. Sometimes I try to take more native solutions (eg. Tox, docker) but I always end up wrapping those tools in a Makefile somewhere forthe road since there are always missing links and because Make is ubiquitous on nearly every Linux and macOS it is just all you need to get a project started.
Example: https://gitlab.com/internet-cleanup-foundation/web-security-...
Running the 'make' command in this repo will setup all requirements and then run code checks and tests. Running it again will skip the setup part unless one of the dependency files has changed (or the setup env is removed).
shavenwarthog2 2 hrs
In 2019 Makefiles are a useful tool for automating project-level things. Too often webapps will require you to install X to install Y to run producing artifact Z. Since Make is old and baked and everywhere, specifying "make Z" is a useful project-level development process. It's not tied to a language (e.g. Tox) nor a huge runtime (Docker). Make is small enough in scope to be easy, and large enough to be capable without a lot of incantations.
The big downside of Make, alas, is Windows compatibility.
deng 8 mins
The big downside of Make, alas, is Windows compatibility.
GNU Make works fine on Windows. The sources come with a vcproj to build it natively, or you get it from ezwinports. At my dayjob, we have a pretty complicated build with GNU Make for cross-compiling our application to Arm and PowerPC, and it works on Windows, even with special Guile scripts to reduce the number of shell calls which are extremely slow on Windows.
pixelrevision 2 hrs
Will not be a problem for much longer :) https://www.theverge.com/2019/5/6/18534687/microsoft-windows...
pickdenis 2 hrs
The big downside of Make, alas, is Windows compatibility.
You'd have to give me a very compelling reason to support developers who use Windows, when Windows lacks these essential tools. Besides, don't people who develop on Windows live in WSL?
ChristianGeek 2 hrs
Nope. I develop in Python, Java, and Kotlin on Windows and never touch WSL. Make is available natively through Chocolatey (a Windows package installer), but I prefer Gradle.
(I also write code to run on Linux, but still prefer Gradle.)
noir_lord 1 hr
Slightly off topic but what would you suggest for someone who is familiar with build systems but who hasn’t used gradle?
I’m just getting into Kotlin and gradle isn’t something I’ve used before since I’m mostly web, .net til now.
astrobe_ 2 hrs
I've compiled a thing or two with MSYS2.
logicallee 1 hr
The big downside of Make, alas, is Windows compatibility.
Isn't the big problem that you have no idea what it's doing to your system? Also that you aren't expected to be able to undo it. You can read the makefiles, of course, but it seems simpler not to have to. (Just update the necessary packages yourself, to the latest version.)
Forgive me if this is naive of me.
theon144 36 mins
Isn't the big problem that you have no idea what it's doing to your system?
As opposed to what exactly? Any other alternative, e.g. separate shell scripts, "npm run" scripts in package.json, running a Docker image, hell even cmake or other make-like tools - does stuff you don't know about without reading the files either.
bch 58 mins
If you write your own Makefiles you know what they do. They’re not that hard to grok and even hand-rolled Makefile use is (IMO) underrated.
RangerScience 36 mins
We're doing this, and I mostly love it. I haven't found a great way to do code re-use across projects yet, and I'm not super happy with the Make function syntax (but, maybe if it needs a function, I should turn it into a shell script that itself is called by the Make command...).
All in all tho, it's a fantastic place to write down long CLI commands (ex: launching a dev docker container with the right networking and volume configurations) that you use a lot when working on the project.
Our Jenkins pipeline also relies on the Makefiles, literally just invoking make release, which is also pretty awesome.
d0mine 3 hrs
fabfile.py (Fabric) could be used as a Makefile in Python. If you don't ever need to ssh to other machines ti run your tasks, you could use pyinvoke library directly (tasks.py). https://www.fabfile.org/
It is easy to add command line arguments to the tasks, configure them using files (json, yaml), environment variables, to split the task definitions into several modules/namespaces.
bb88 3 hrs
Having used fabric in the past, I've always found it just as easy to use a shell script and make files.
There's always some level of bootstrapping a project (installing packages/software, compiling libraries and dependencies) where it's easier to just to write a shell script than to program python to do. E.g. How do you get fabric installed on a system?
There's also been this longevity of sorts that Make seems to have gotten right. People just keep going back to it because it's simple.
theossuary 2 hrs
I've been moving away from using shell scripts in a tools/ directory to using Python Invoke (http://www.pyinvoke.org), which is the library underlying fabric.
I used bash scripts for years, but for a lot of reasons made the switch:
-
It was always painful to create small libraries of functions used across multiple scripts in a project
-
It's difficult to consistently configure them with per-user settings. I've written bash implementations of settings management, Invoke handles this for me.
-
I'd still have to reach for Python whenever I needed to do anything with arrays or dicts, etc.
-
Getting error handling correct can be a chore
Invoke has a lot of nice to haves to:
-
Autogenerated help output
-
Autocomplete out of the box
-
Very easy to add tasks, just a Python function
-
Easy to run shell code when needed
-
Very powerful config management when needed
-
Supports namespacing, task dependencies, running multiple tasks at once and deduplicating them
It's not perfect, but it's a lot better than my hand rolled scripts were.
Groxx 2 hrs
it's also far, far, far easier to make it work predictably on multiple platforms. and easier to understand and change later. that can get nightmarishly hard in make/bash, once you go outside the super-simple realm.
anewhnaccount2 2 hrs
Snakemake is a quite nice improvement on make for data munging stuff.
pletnes 3 hrs
One of the worst problems with using windows (in my opinion) is that there’s no native GNU make.
deng 14 mins
One of the worst problems with using windows (in my opinion) is that there’s no native GNU make.
GNU Make even comes with a vcproj file for building a native binary with Visual Studio. Worked fine for me. Building it with Guile support though is difficult, but fortunately Eli Zaretskii provides native binaries through his ezwinports, and they worked pretty much flawlessly for me. Of course you will usually need a shell to execute recipes, but Make itself runs natively. For more information, see README.W32 in the sources.
shadowfox 57 mins
There are a number of ports of GNU Make to Windows. MSYS2 [1], for examples, provides a reasonable development environment that includes Make.
If you just want a Make, there is [2] which can be installed separately and is part of the GNUWin32 collection.
[1] https://www.msys2.org/ [2] http://gnuwin32.sourceforge.net/packages/make.htm
aequitas 3 hrs
Isn't non-native development on Windows a solved problem nowadays with WSL(2)?
murderfs 3 hrs
WSL is currently horrendously (unusably, IMO) slow. WSL2 promises a 20x speed up, but it was already 100x slower than native Linux at some actually-realistic workloads that happen all the time when you're developing (e.g. git grep), so it's probably still too slow to be tolerable.
I had the opposite problem of wanting to develop some stuff for Windows from a Linux environment, and I settled on running a linux VM and copying binaries over by scping to WSL, which works reasonably well.
pletnes 1 hr
A nice thing with WSL is that you get working make and rsync. But I would like make for coding on native windows. Many FOSS projects use Makefile as the parent post described.
trendoid 1 hr
Any good resources for learning about make files that you can recommend?
aequitas 3 mins
I don't really know a good all in manual, most thing about Make I learned over years of using it from different sources. And I still sometimes discover new features (and new ones are still added in recent release, but I tend to avoid them to keep Makefiles compatible on older systems).
But the Make manual is pretty comprehensive as a guide and reference: https://www.gnu.org/software/make/manual/make.html
Also (as with most things) knowing what name some concept has makes it easy to search for references. For example the terminology of rules (target, prerequisite, recipe): https://www.gnu.org/software/make/manual/make.html#Rule-Synt...
Things I tend to google often because I forget and some are used more often than others are: automatic variables, implicit rules, conditionals and functions.
One trick that really helps making Make complete is making your own pseudo state files and understanding the dependency system. One of the best features of Make is its dependency resolving. You generally write rules because you want a target (a file or directory) to be created, based on prerequisites (dependencies) according to a recipe (shell scripts). Make figures out that if the prerequisites didn't change, it doesn't need to run the recipe again and it will reuse the target. Greatly saving on build time.
Because Make relies on file timestamps to do its dependency resolving magic if you don't have a file there is not much Make can do. So what you can do instead is create a pseudo target output yourself. For example: https://github.com/aequitas/macos-menubar-wireguard/blob/mas... Here a linter check is run which creates no output. So instead a hidden file .check is created as target. Whenever the sources change the target is invalidated and Make will run this recipe again updating the timestamp of .check. Also note the prerequisites behind the pipe (order-only prerequisites). These don't count toward the timestamp checking, but only need to be there. Ideal for environment dependencies, like in this case the presence of the swiftlint executable.
deng 5 mins
The GNU Make manual is excellent.
For learning advanced techniques: "The GNU Make Book" by John Graham-Cumming.
ashton314 7 mins
This video helped me:
https://www.youtube.com/watch?v=fkEz_oVh0B4
antipaul 13 mins
https://github.com/pavopax/gists/blob/master/makefile-quick-...
gilmi 1 hr
Matt Might's article is really good:
http://matt.might.net/articles/intro-to-make/
messe 46 mins
Worth noting that that's an introduction to GNU make, which, while the most common implementation, isn't the only one out there.
kyllo 4 hrs
Learning in-depth your various options for persisting data, is very useful since most applications have to deal with persistence in some form, and increasingly in a distributed manner. Go beyond simply skimming the surface of SQL vs. NoSQL and the marketing claims different databases make about their scalability and consistency. Learn what ACID and CAP stand for and the tradeoffs involved in different persistence strategies. Learn SQL really well. Learn how to read a query plan, which is the algorithm your SQL query gets compiled into. Learn about the tradeoffs of row-based vs column-based storage. Learn how indexes work, and what a B-tree is. Learn the MapReduce pattern. Think about the tradeoffs between sending code to run everywhere your data is stored vs. moving your data to where your code is running.
theossuary 4 hrs
Two great resources I've been going through are
hypertexthero 1 hr
Another good resource that guides one through both the philosophy (why) and technical details (how) of building a web application is Software Engineering for Internet Applications:
https://philip.greenspun.com/seia/
barbecue_sauce 1 hr
Designing Data-Intensive Applications is probably the best O'Reilly (if not overall technology) book of the past decade.
Robin_Message 4 hrs
Read the curriculum of an undergraduate computer science course and read up on the things you haven't heard of. Some courses will even have lecture notes available.
E.g. these four pages are the university of Cambridge masters in computer science:
https://www.cl.cam.ac.uk/teaching/1819/part1a-75.html
https://www.cl.cam.ac.uk/teaching/1819/part1b-75.html
https://www.cl.cam.ac.uk/teaching/1819/part2-75.html
https://www.cl.cam.ac.uk/teaching/1819/part3.html
(Or a MOOC, but the links above are easy to browse text, syllabuses and lecture notes, not a load of videos.)
JoeMalt 58 mins
To clarify: the first three links are for each year of the (three-year) undergrad program, the fourth is for the Masters.
The Cambridge course isn't perfect, but they do a very good job of making as much teaching material as possible publicly available.
rat9988 2 hrs
Thank you very much. This is very valuable.
gwbas1c 3 hrs
Unit testing, mocking, and various other testing techniques.
Why? Any project of sufficient complexity is very hard to test. If all you're doing is code -> build -> run to debug your code, you can very easily break something that's not in your immediate attention.
The problem is that good unit testing is hard, and time consuming. It can be so time consuming, that unless you can really plan in advance how you test, you could spend more time writing test code than real application code. (This is what happens when writing professional, industrial-strength code.)
So, when a hobby project becomes sufficiently interesting enough; such that the code will be so complicated that your code -> build -> run loop won't hit most of your code, you should think about how to have automated tests. They don't have to be "pure, by the book" unit tests, but they should be an approach that can hit most of your program in an automated manner.
You don't need to do "pure" mocking either. If you're writing something that calls a webservice, you could write a mock webserver and redirect your program to it. If you're writing something that works with pipes, you could have a set of known files with known results, and always compare them.
The goal is that you should cover most of your program with code -> build -> tests; and only do code -> build -> run for experimentation.
wpietri 1 hr
Let me second this. And in particular, I strongly encourage every developer to try starting a new project in a test-driven fashion (by which I mean that you advance the code by writing a bit of test and making it pass, and then doing that over and over.)
There's a qualitative difference between working in a well-tested code base that's very hard to describe convincingly. A lot of my early development experience was in code bases that had little or no testing. Experiencing a well-tested code base totally changed things for me. Instead of work being a death-of-a-thousand-cuts experience, it became pleasant, steady progress.
epiphanitus 20 mins
Are there any resources out there you would recommend for learning testing techniques in a Python context?
nickjj 53 mins
Shell scripting for processing text. You can often get so much done with so little code and effort.
Also on a semi-related note, I think as a self taught programmer, it's easy to get stuck on things that seem cool but are just procrastination enablers (I know, I've been guilty of it for 20 years). Like, if you're about to start a new project and you want to flesh out what it's about, you really don't need to spend 5 hours researching which mind map tool to use. Just open a text document and start writing, or get a piece of paper and a pen. It won't even take that long.
I spent about 1.5 hours the other day planning a substantially sized web app. All I did was open a text file and type what came into my head. For fun I decided to record the whole process too0. I wish more people recorded their process for things like that because I find the journey more interesting than the destination most of the time. Like your journey of eventually finding message queues must have been quite fun and you probably learned a ton (after all, it lead you to message queues, so it was certainly time well spent).
Draiken 33 mins
This 1000x. I put of just getting by with shell script for some years and when I finally decided to get deeper into it, it's magical.
A good series of piped commands with tools available basically everywhere can solve problems you had no idea could be so simple to solve.
0x262d 25 mins
have any good resources for this?
fancyfish 1 min
Perhaps these are a good start:
awk: - GNU Awk User Guide - https://www.gnu.org/software/gawk/manual/gawk.html#Getting-S...
sed: - Grymoire guide - https://www.grymoire.com/Unix/Sed.html - Official docs - http://sed.sourceforge.net/#docs
bromonkey 1 min
man bash
man awk
man sed
man grep
jimpudar 3 hrs
Learning how to use dtrace / bpftrace 0 is very valuable if you ever need to get into serious systems profiling.
There are some really cool data structures out there you might not know about. One of my favorite basic ones that I get a lot of use out of is the trie [1] (a.k.a. prefix tree). Very useful for IP calculations.
Also look into probabilistic data structures [2], very amazing things can be done with them.
0 https://en.wikipedia.org/wiki/DTrace
[1] https://en.wikipedia.org/wiki/Trie
[2] https://en.wikipedia.org/wiki/Category:Probabilistic_data_st...
cryptonector 3 hrs
DTrace is life-altering.
I keep hoping that someone will build a dtrace(1) CLI that transpiles to bpftrace.
stcredzero 2 hrs
Profiling, period
bradford 8 mins
I've extensively used Diff tools as an analytical aid across many different domains. It has uses far beyond code change tracking!
Specifically, I like vim's interactive diff'ing capabilities (although any interactive diff tool with sufficiently powerful text-editing capabilities should suffice).
So much of the troubleshooting that we do in programming is asking "this thing used to work, what changed?". Don't rely on your eyes to find the changes, let the computer do the work for you. The ability to load up two different log files in a diff session, regex-substitute the noisy portions away (dates, process/thread id), and view the key functional changes really helps me go from noise to signal in an optimal way.
m_fayer 4 hrs
https://dataintensive.net/
I can't recommend this book enough. I have a CS background, and still had quite a few "I can't believe this thing has been hiding in plain sight!" moments while reading it.
redisman 4 hrs
It's great. Incredibly dense with useful information and it just blows my mind how much knowledge Martin has about the topic. I recommend watching this talk from him to give a little glimpse of the book: https://www.youtube.com/watch?v=5ZjhNTM8XU8 This is just about a little part of one of the chapters.
mirekrusin 1 hr
Oh man, this is good, thank you.
copperx 4 hrs
I'm now torn between reading this one first or the Architecture of Enterprise Applications.
jb3689 3 hrs
I loved Designing Data-Intensive Applications. It gives you the reasons why NoSQL databases exist and the problems they solve. Moreover it gives you reasons to select one over another. It's really excellent and one of my top two CS books
politician 4 hrs
If it helps, IMO "Designing Data-Intensive Applications" is a better bang-for-the-buck. Enterprise-scale applications are a world unto themselves.
copperx 2 hrs
Edit: I meant Patterns of Enterprise Application Architecture by Fowler in my comment above. Recommended by DHH.
louthy 1 hr
My advice would be to skip it completely. It's just packed full of standard GoF OO dogma.
nerpderp82 4 hrs
Debuggers and property based testing. It is a select few people that can actually productively (not their own metrics) use print statements for debugging. Learning how to craft repro scenarios and adequately capturing state in a debugging session can enable junior devs to easily surpass senior devs.
Property based testing aren't quite formal methods, but I think they are a good stepping stone. And they also somewhat force your code into an algebraic/functional style which also make it amenable to refactoring, better testing and is easier to understand.
Design tools like Swagger can help one think through services w/o diving into code. Code itself is a liability and should be thought of as "spending" not creating. Code is a debt.
Refactoring and code understanding tools, if you use PyCharm (you should, it is free in all senses), learn how to navigate into your libraries. Read you libraries.
pankajk1 21 mins
Here is a list of technologies/tools that I have found quite enabling (in addition to what has already been covered):
-
Earlier use of VMs and now Docker containers -- It was around 2004 that I got introduced to VMware tools to create, configure and run VMs. Life was never the same after that. No more fretting about installing pre-beta software for the fear of hosing my perfectly working system. There was a time when I had multiple servers running VMWare hypervisor, each running multiple systems. Then around 2014 I switched to using Docker containers for similar purposes and haven't had a need to use VMs.
-
Jupyter notebooks/pandas/plotly -- Can't imagine how one can explore data without this.
-
SQLite -- Perfect for writing unit tests for code that deals with SQL databases.
agentultra 2 hrs
Formal methods.
It took me nearly a decade of working in distributed systems to be introduced to TLA+ and other tools in this space. Until then my knowledge had been built from textbooks describing the fundamental data structures, algorithms, and protocols... but those texts take an informal approach with the mathematics involved. And since I was self-taught I was reading those texts with an eye for practical applications than for theoretical understanding. I had no idea that a tool existed that would let me specify a design or find potential flaws in systems and protocols, especially concurrent or parallel systems, with such ease.
I think type theory and category theory have also been great tools to have... but I think mathematics in general is probably the more useful tool. Being able to think abstractly about systems in a rigorous way has been the single-biggest booster for me as a practitioner.
pickdenis 2 hrs
the single-biggest booster for me as a practitioner.
Can you instantiate this claim with an example? I'm somewhat knowledgable in both math and computer science theory but have yet to feel as though my math background has helped me in practical CS.
hsitz 1 hr
A new version of "The Pragmatic Programmer" recently came out. [EDIT: not available yet, only preorder at amazon, beta version available at pragprog.com.] That book is all about tools and methods that a self-taught programmer should look into:
https://www.amazon.com/Pragmatic-Programmer-journey-mastery-...
colomon 1 hr
For me, that Amazon page is listing it as a pre-order, without any release date. And all the other versions (Kindle, Paperback) are the 1st edition instead of the 2nd.
Very frustrating, as I considered the first edition to be essential and upon reading your comment, instantly went to purchase the 2nd edition.
Edited to add: Found a date, Amazon is listing it as October 21, 2019.
hsitz 4 mins
Sorry, I thought I'd read a review of it already, so just didn't look closely to see it wasn't available yet.
It looks like you can get a DRM-free beta version of the ebook on their website, with free upgrades to published version once it's finalized: https://pragprog.com/book/tpp20/the-pragmatic-programmer-20t...
RangerScience 23 mins
At the job I'm at, I've picked up three tools either for the first time, or in a very new way:
-
Makefiles. See @aequitas' comment for more.
-
Terraform. Seriously, just using this tool taught me [a lot of] devops. It's fantastic!
-
Docker (as a tool!)
I'm going to go into the third one a bit - I feel like Docker is mostly thought of as useful for deploying things to the 'net (kubernetes, ECS, etc), but I think it's also amazing for local development and build pipelines. I actually have no idea one way or the other how much other people use it this way as well, so maybe it's just me that's finding it unexpectedly awesome for this.
Put together the right CLI command + Dockerfile, and you can hand someone a repo and they can launch a complete, reliable development environment in a single command without any other system prep. No more worrying about which dependencies need to be installed in what way; it's like rvm + bundle exec but for EVERYTHING. No more dealing with whatever custom system modifications someone has going on. git clone, make dev move on with life.
And then you can also have Dockerfiles that are specifically for producing your build artifacts, and then completely ignore the container for execution. This is how I'm using both AWS EMR and Lambda.
enkiv2 3 hrs
I highly recommend learning PROLOG & understanding how to write your own simple planner system. The hairiest real problems are hairy because they're best suited to a declarative style (and programs written declaratively can be made much more efficient through more clever solvers -- given naive code, a clever solver has a much bigger efficiency boost over a dumb solver than an optimizing compiler does over a non-optimizing one -- although PROLOG itself leaks too much abstraction for many of these techniques to be viable in it).
I also recommend understanding message routing systems used in file sharing, like CHORD.
If you don't have a strong background in the math behind theoretical computer science, you might benefit a lot from an understanding of the formal rules around boolean logic, symbolic logic, & state machines -- especially, rules about when certain kinds of things are equivalent (since stuff like demorgan's law are used for simplifying and optimizing expressions a lot, and rules for state machines are used to prove upper limits on resource usage).
If you don't already, learn to use awk. It's a much more powerful language than it seems, and fits extremely well into the gap between command-line prototyping in shell one-liners & porting a prototyped tool to python or perl, and so it's a huge time saver: it is faster to write many kinds of tools in a mix of shell and awk and then rewrite them in python than it is to write them in python in the first place.
bdamm 3 hrs
I've never used Prolog in the 15 years since I learned it in college. It's an interesting take on programming, for sure, and I appreciated the mind-expanding exercise, but hasn't helped me in my career at all.
Totally agree on awk. I use it almost every day for quick little one-liners. Big time saver.
Also agree on state machines, because from there it is a short hop to understanding formal grammars and the foundation of compilers and languages, which has been immensely useful in my career.
Jach 2 hrs
If you're strictly asking about things a self-taught hobbyist programmer may have missed, then I second the suggestions here to skim through a CS degree curriculum and dig into anything that's unfamiliar. As one possible example, maybe you're solving a problem trying to parse some text, and you're in over your head with ad-hoc regexes and conditionals and type casts and exceptions all over the place. If you've seen the concept of a grammar (likely to come up in CS programs, though not all) and of generating parsers / validators for them, you can eliminate a whole lot of programming by specifying a grammar in some common format (for instance, ABNF) and running it through a program that generates a parsing program for you. The general category of programs writing programs is worthwhile to look into, and playing with languages where such a feature is first-class (like Common Lisp, which apart from having macros also has a compile-file function you can invoke at runtime) can be enlightening.
A lot of the comments here are highlighting things that not even most CS degree holders will know about, nor many professional programmers. Those can be useful for a hobbyist too, so maybe the lesson is that regardless of your current level of knowledge it's important to keep in mind: "There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy."
Your comment about feeling comfortable in the practical details reminded me of how I used to feel as a fresh self-taught PHP-wielding teenager after a few years of it. As an "expert" PHP coder, I could do anything! Even an OS if I wanted! Well I learned better eventually. :)
stormking 58 mins
"If you've seen the concept of a grammar (likely to come up in CS programs, though not all)"
Really? That was one of the very first topics in my first semester. How do you teach CS without grammars and the Chomsky hierarchy?
Const-me 23 mins
I don’t think there’re many ridiculously useful things that are universally useful.
If some stuff is domain specific or have much theory involved doesn’t mean it’s not considered a fundamental concept in some areas of work.
If some other stuff was very useful for you doesn’t mean it’s universally applicable. For example, if you had worked on a high-frequency trading bot, I don’t think you would have used neither Python, nor ZMQ or other general-purpose messaging middleware, nor even OS-provided TCP/IP stack — they all cause too much latency.
I’ve been programming for living since 2000, worked in a lot of different stuff from web dev and enterprise to videogames, embedded, robotics and GPGPU. Yet I can name many huge areas which I hadn’t seen close enough, or at all, along with libraries and tools used by people working there.
Every time I start working in a new area, or when I resume working in an area after a long (years) pause, I read a lot of relevant stuff. Continuous learning is the key to stay good, IMO.
mnemotronic 1 hr
* Regular expressions. The most valuable "not a language" that I know. * SQL. For me, it was a trip coming from a procedural language background. I kept trying to figure out how to do loops. * Command line - DOS and unix * Batch languages for those. * HTML / CSS / Java (please don't kill me) Script and the DOM
cameronbrown 1 hr
I'm surprised nobody's mentioned spreadsheets - specifically Google Sheets or something scriptable and hosted. Recently I've built up a small system which sucks in data from a few places (fitness, task management, calendar,..) and analyses it against several goals I've set. This means I can see how I'm progressing towards what I want without even touching it anymore.
They're a really nice UI for bootstraping projects, and even some small databases. A current project of mine for a client uses sheets as a backing store for an email collection list. Since there's only a few hundred rows, it makes sense at this scale and works really well for non-technical users.
Forge36 1 hr
Similarly Microsoft Access. I learned Access for my first job (100% of my job was moving tasks into Access, 3 years later I helped some interns transition to a SQL server+web application). 10 years later I'm still using Access for some aspects of my new job. The ability to quickly prototype something and receive almost immediate value is vastly underestimated (I know of one company which runs all of their tasks out of Access).
cameronbrown 1 hr
I suppose what I've described is just a little more of an unstructured version of this. Mixing data and computation in such a way is definitely a tech debt tradeoff though.
vosper 4 hrs
Your IDE, its refactoring tools, but especially its debugger.
VSCode or PyCharm (assuming you are still a Python developer) could be a good place to start. I'm always surprised when I see professional developers coding in Sublime Text and debugging with print statements (or their equivalent). Usually you have better options than that, especially for statically-typed languages - but even for JS and Python.
niklasjansson 1 hr
Excel. Seriously. For more things than you imagine
Let's say you want to make a property assignment for some class;
this.a=x.a this.b=x.b ...
While you probably would want to do this some other way to start with, and while you of course can solve it using some emacs wizardry, I can whip that up in Excel using some formulas in a matter of a minute. Moreover, I can keep adding to it when I realize something was missing.
I can make a diff, join or union of the results of two queries from different databases (or even database engines). Not to mention calculation and design mockups.
slang800 1 hr
Alternatively, use Jupyter Notebooks & Pandas for this type of work. It has the same interactivity & visualization as Excel, but is far more powerful and you can reasonably move the code you create into your final product, rather than rewriting Excel formulas into a proper programming language.
I've heard that Airtable occupies a similar space, but I haven't used it enough to recommend it.
hsitz 1 hr
I'm not following. What good is it to have equivalent of "this.a=x.a this.b=x.b ..." in Excel, when what I need is to have this in code in my editor? Are you saying you use Excel to create the code, then copy/paste it into your editor? Or what?
JonathonW 58 mins
I think the parent's using Excel essentially to do repeated template expansion: e.g. for a given set of member variable names ([a, b, c...]), give me the assignment statements I'd need to use those in a constructor.
Which I could do pretty trivially in Excel... but could also do trivially in about two lines of Python:
vars = ["a", "b", "c"]
statements = ["this.{0} = x.{0}".format(var) for var in vars]
print(statements)
Spreadsheets can be an incredible tool-- as an interactive environment that allows non-programmers to express domain knowledge and quickly automate parts of their workflow, they're really unparalleled. But this isn't a very good example of something that Excel is particularly well-suited for.
WoodenChair 1 hr
The best tools you can teach yourself are arguably more fundamental than any specific language or library, but instead computer science problem solving techniques. I've written books on this subject specifically targeted towards self-taught programmers: https://classicproblems.com/
Sorry for the self-plug, but it's super relevant I think.
sycdan 14 mins
I use ack (https://beyondgrep.com/) many times per day, and it has saved me a great deal of time. That said, if you use IDEs (I don't), it may be somewhat redundant.
jacksnipe 2 hrs
Not nearly as big a deal as some of the other tools and techniques being mentioned, but tmux/screen are lifesavers when you need them.
mattnewport 4 hrs
If you're a self taught hobbyist you may not have had much structured exposure to fundamental data structures and algorithms and complexity analysis. I think that type of thing is easier to learn when you already have some experience so you can relate it back to real world problems you have encountered as you describe doing here. Now might be a good point in your development to dig into some of those fundamentals if you have not done so much in the past.
Khelavaster 20 mins
Visual Studio, especially its visual UI designer, build & reference management, and scaffolded website development (ASP.Net MVC with Entity Framework from scaffolding). Night and freaking day.
Also, WCF for all your inter-computer communication needs. I wouldn't be developing software today if I weren't introduced to low-trouble application development with Visual Studio
throwaway55554 3 hrs
Develop a project that has two local processes that have to communicate, one local database, and one remote service that the two processes have to communicate with.
Doesn't matter what the project is. Could just be tossing around a string of data that eventually gets dumped into the db and sent to the server.
Research different methods of architecting such a system. Code up a few.
By doing this, you will actually find the answer to the question you posed. And, arguably, have fun doing it.
augbog 2 hrs
Honestly I still find a lot of engineers don't know git properly. Like they know enough to commit and push but that's about it. It really helps to understand everything git has to offer.
RangerScience 32 mins
Absolutely. When I get people into learning Git, I start with the obvious (checkout, commit) move to the essential (branches, forks) and then usually say "When you're comfortable with that, start playing around with rebase. When you get into trouble with that, come talk to me again, because that's the key learning moment."
I've learned more messing up when using fancy git rebase stuff than any tutorial.
333c 2 hrs
The three most important basic git operations to know (in my opinion)
git checkout -b
git log
git rebase -i
btilly 1 hr
I strongly prefer git merge over git rebase.
Using rebase results in a cleaner history and simplified workflow in many cases. However it also means that when you have a disaster, it can be truly unrecoverable. I hope you have an old backup because you told your source control system to scramble its history, and you don't have any good way to back it out later.
For those who don't know what I mean, the funny commit ids that git produces are a hash signifying the current state of the repository AND the complete history of how you got there. Every time you rebase you take the other repository, and its history, and then replay your new commits as happening now, one after the other. Now suppose that you rebased off of a repository. Then the repository is rebased by someone else. Now there is no way to merge your code back except to --force it. And that means that if your codebase is messed up, you're now screwed up with history screwed up and no good way to sort it out.
That result is impossible if you're using a merge based result. The cost is, though, that the history is accurately complicated. And the existence of a complex history is a huge problem for useful tools like git bisect.
jonahx 35 mins
Linus on rebase:
https://www.mail-archive.com/dri-devel@lists.sourceforge.net...
333c 1 hr
I work on a small open source project. Our master branch is protected from force pushes, and I only use rebase on personal branches for features or bug fixes. It's very nice if I'm planning to merge several commits but I need to fix something in one or more of them.
I agree that rebasing should not happen in the main branch(es) of a repo.
bshacklett 6 mins
git add -p changed the way I think about commits
333c 0 mins
I love that one too! It's great for creating commits that address one specific thing.
dwoot 1 hr
I'll have to add git reflog for instances where you've felt that you've completely screwed up something as one can always move back to a previous state. I think this is essential.
A useful one that I'll add is what I call the sword command: git log -S<word> This one allows one to list commits that contain a particular change. This has been useful in tracking down old changes
jimpudar 1 hr
I also find git blame extremely useful, along with code exploration tools like DeepGit 0
0 https://www.syntevo.com/deepgit/
azhenley 13 mins
Usability engineering: https://en.wikipedia.org/wiki/Usability_engineering
joker3 3 hrs
https://ocw.mit.edu/courses/electrical-engineering-and-compu...
In my experience, most hobbyist programmers are fine writing scripts and other small programs but have no exposure to the sort of ideas that are necessary for making large programs. This course at least exposes you to a lot of those concepts.
metahost 1 hr
The latest course is available here: http://web.mit.edu/6.031/www/sp19/
Khelavaster 19 mins
It's absolutely critical to learn object-oriented programming in a mature language like C#. (Java's OO model is slightly broken, as well as fundamentally underdeveloped; it's better tackled after learning a correct, functional object0oriented language like C#.)
fancy_pantser 4 hrs
The next obvious step after message queues and distributing work would be streams.
Here is an excellent introduction to unified logs and stream processing by the author of Kafka at LinkedIn:
https://engineering.linkedin.com/distributed-systems/log-wha...
wpietri 1 hr
For me a lot of programming involves some amount of data wrangling. E.g., getting input data ready. Or generating and understanding results of some technical experiment. I recently came across VisiData and adore it. It has a steep learning curve, but I've found it very much worth it: http://visidata.org/
CalChris 3 hrs
Godbolt Compiler Explorer: https://godbolt.org/
DTrace: http://dtrace.org/
SonOfLilit 1 hr
More important than any specific technique or tool is that you get an experienced mentor to look at your work once in a while and point out the obvious things relevant to the specific project that you don't know you don't know. Seriously, you'll probably save a few months of work in the first ten minutes of talking to a senior developer about your hobby project.
Talk to someone with many years of experience, though, otherwise you'll most likely get sent on a quest for beauty of implementation that has nothing to do with the goals you're trying to achieve. Sadly, that is a lesson that takes a long time to internalize.
SonOfLilit 1 hr
(Learning all the things recommended in this thread would take a few years, and maybe half of them would be useful given you know them, a lot fewer useful enough to justify the price of learning them)
namank 33 mins
What you want to do is ponder over how common functionality around you must work - if you can't come up with a reasonable solution, it's time to google it and learn it. If you can come up with a reasonable solution, google to check if you're correct.
This is the most pragmatic way I've found to learn and stay in touch.
neilv 3 hrs
At some point, it helps to broaden your programming language exposure, even if you stick to mostly one language for most of your work. You'll find ways to apply ideas from other languages/communities to your work.
Try to spend some time learning idiomatic programming from one of the Lisp family (Scheme, Racket, CL, Emacs Lisp, and Clojure all have different thinking, but a lot of overlap). Play a bit with Smalltalk or a similar descendant, even if you're already doing OOP elsewhere. At some point you should learn a textual expansion language, like one of the Unix shell scripting ones, or Tcl (and learning basic Bash scripting will probably be useful in tech work). Try a logic programming language, like Prolog, or one that's a minilanguage within another, like Mini-Kanren. Maybe buckle down for hardcore functional programming (e.g., Haskell, OCaml, or discipline yourself to do it in a Lisp?). You should also get comfortable with C or at least an assembly language at some point, to have a better idea of what other languages are and aren't giving you, and also C is just a really useful thing to know when you need to write a little fast code, FFI to a native library, or get into languages/IRs for newer target architectures.
(Disclosure: I've been especially involved with Racket, an energetic close descendant of Scheme, and have some interest in promoting it, but I'd list a Lisp as one of the first in any case.)
jimpudar 1 hr
Racket is a very good Lisp for beginners, DrRacket makes it easy to get up and running with little effort.
neilv 1 hr
Agreed. Racket is from a particular school of thought, and you won't get all Lisp family ideas from it, but it's great.
You can start up DrRacket (a simple IDE for students that can also be used for professional work, and has some powerful features in it), or just use your favorite editor and the racket command-line program and REPL. There's way too much documentation at: https://docs.racket-lang.org/
You can also do the old MIT introductory CS textbook, SICP, using Racket.
snazz 1 hr
You can also do the old MIT introductory CS textbook, SICP, using Racket.
#lang sicp
https://github.com/sicp-lang/sicp
lixtra 3 hrs
Learn to use a profiler for your programming language. It will tell you what’s worth optimizing in your programs. I.e. if your program spends only 10% of the time talking to others than that may be the maximum you can optimize by choosing a fancy communication pattern.
paulgb 4 hrs
State machines. They can be a helpful abstraction for UI or business logic, and they also make regular expressions make a lot more sense.
jimpudar 1 hr
Yeah, far too often people create horrific spaghetti code of if/else statements which could be neatly coded as a state machine. This is a very important concept to grasp.
LeonB 30 mins
So many of the problems are “people problems” so in addition to the excellent technical suggestions in this thread I’d add books like “Thanks for the Feedback”, “How to Win Friends and Influence People”, “Getting to Yes”
narag 3 hrs
A few decades ago, a new programming environment exploded: the web. Looking for a ridiculously useful tech stack? Look no further: HTML, HTTP architecture, SQL backend... a guide written at the time:
http://philip.greenspun.com/panda/
Paul Graham also wrote about why the web was such a deal, IIRC in the Beating the Averages essay. In particular: you can use whatever tools you want and avoid deploying to client machines.
cryptonector 3 hrs
PostgreSQL + PostgREST + react-admin == fantastic stack.
You can write an entire application in SQL and PgPlSQL but using an HTTP JSON API as the interface with a static and responsive BUI.
This allows you to be extremely agile in development and ops (because, e.g., you get to use logical replication).
I can't say enough good things about this approach.
davb 23 mins
This reminds me of how we used to create thick client desktop software, connecting directly to the database. Doing most of our business logic on the front end and leaning on the database for auth, relational constraint enforcement and (via stored procedures) transactional consistency and validation. It felt weird at the time when the whole world seemed to move to a three layer model with app servers sitting in the middle - now it's hard to imagine it any other way.
pandemic_region 2 hrs
indeed if your app is about managing your personal dvd collection, or variants thereof, such anemic UI-to-database tools work very well. Not when there is complicated domain logic involved.
cryptonector 2 hrs
Having built such an application (w/ complex business logic), I have to disagree. But I'd like to hear what problems you've run into.
I would agree that react-admin is a bit too simple, and this stack really calls for a BUI to be built specifically to fit the PostgREST model.
theonemind 1 hr
* Parser generator tools like ANTLR https://www.antlr.org/, or even lex and yacc, useful for parsing languages/config files and probably generating a better/more robust parser than you'd cobble together by hand
- Dynamic programming https://en.wikipedia.org/wiki/ -- great for relatively-quickly (in computation time) coming up with good-enough solutions to some hard (like NP hard) recursive problem that would take forever (almost literally) to technically find the absolute best solution to, but not something you'll use every day.
oneturkmen 6 mins
That's weird. DP, though better than naive brute force, is still very slow since it looks only for an exact, optimal solution.
sgillen 43 mins
I was under the impression DP was usually used for getting the exact answer to certain classes of problems faster than exhaustive search. I usually see approximations compared to the ground truth dynamic programming answer.
Do you see the reverse in your work?
brianpgordon 4 hrs
Don't go too overboard with message queues. There's nonzero development and operational overhead incurred when part of your application takes its input in a weird binary format, and when the data in your queue is thrown away after processing, and when you need to think about scaling of workers and concurrency. If you're not working with real "big data" – and, let's be honest, almost nobody is – I would advise using an HTTP-based service (REST, SOAP, whatever, take your pick) for communication and a SQL/NoSQL/NewSQL database for state.
jrockway 3 hrs
ZeroMQ is not really a message queue, it's more of a networking library. It takes TCP sockets and adds other concepts on top, like request/reply or publish/subscribe.
massaman_yams 3 hrs
Doing a task queue in an RDBMS is not as simple as some might think, though. https://www.2ndquadrant.com/en/blog/what-is-select-skip-lock...
quanticle 4 hrs
I would recommend learning the SOLID [1] design principles. I've found them to be a very helpful guide when designing software components.
[1] https://en.wikipedia.org/wiki/SOLID
zoomablemind 4 hrs
I'm not sure if 'hoarding' such knowledge would be practical.
Sometimes, having a limited toolset would better focus you on the problem at hand. Then, once the challenge is clarified, the search for alternative ways to architecture and implement it would become practical.
If just for fun of exploring something new, pick whatever interests you in general, language, framework, a domain. Async processing and idioms?
amrox 4 hrs
Learning the basics about how programming languages work - parsers, interpreters/compilers. I've heard good things about Writing An Interpreter In Go [1]. Related, I've enjoyed Martin Fowler's DSL book [2].
gaze 4 hrs
Get good at math. It'll serve you well and never go out of style.
ThrustVectoring 4 hrs
Math is a bit too broad. From personal experience, the most relevant topics for programmers are Linear Algebra and Discrete and Combinatorial Algebra.
gaze 2 hrs
Eh I mean, I think there's a certain discipline that comes with studying any branch of math to a certain degree of rigor. But yes, linear algebra and discrete math are probably the most useful. I think control theory and optimization are under appreciated amongst programmers though.
lostphilosopher 4 hrs
I'd recommend statistics specifically. Comes up everywhere and it's easy to be wrong about if you don't dig into it.
wainstead 3 hrs
Second this, if you only work through the Khan Academy stuff on the various things like scatter plots, standard deviation, the normal distribution, etc. you'll be much better off for it.
hannob 3 hrs
If you happen to program in C/C++ you should absolutely familiarize yourself with the sanitizers of gcc and clang, most notably address sanitizer. (However there are good reasons why you shouldn't program in C/C++ to begin with.)
OliverJones 3 hrs
Event-driven, asynchronous, programming
Here's a valuable addition to your toolkit of mental models for programming: Event-driven, asynchronous, programming (in the style of ES6 Javascript or a similar language. )
Some suggestions about how to learn the basics? tutorials on...
-- building a so-called "single page web app" with a framework like vue.js or even jQuery.
-- node.js to build a complex back-end server without using a threading model)
-- React (to build an interactive program to run in a browser)
caust1c 2 hrs
Heh, funny that you mention that ZMQ is overlooked. While often though of a message passing library or serverless queue, Zero MQ has some pretty severe limitations that implementers fail to consider.
To be clear, I think it's an amazing library which is unmatched in its performance, but it comes at a cost: reduced reliability.
ZeroMQ will drop messages in a number of situations. The library does not handle delivery guarantees which means that the application must do it themselves. Whether or not this works for you is an application level concern. However, having used it at two companies now: both times it ended up being thrown out for a more reliable queue (kafka).
http://zguide.zeromq.org/py:all#Missing-Message-Problem-Solv...
So maybe the reason you haven't stumbled upon it sooner is because it's overhyped? Definitely useful but with a grain of salt.
wallstprog 2 hrs
The "MQ" part of the name is unfortunate, but apparently came about because of the original idea to come up with a "better" implementation of AMQP (http://zeromq.org/docs:welcome-from-amqp).
But you're right -- ZeroMQ doesn't do queueing (except in some very limited circumstances), and if you need reliable delivery you must implement that yourself "on top of" ZeroMQ. I've done that, and while it's not a simple task, it is certainly possible.
You can get reliable delivery "out of the box" with other software, that in fact does do queueing. (kafka may be one, but I don't know enough to say).
But what you give up when you do that is performance -- ZeroMQ can easily be orders of magnitude faster than those other solutions, and for some applications (e.g., real-time market data) the work to provide a custom reliability solution on top of ZeroMQ is worthwhile.
aasasd 2 hrs
Yeah, every new user has to learn that ZMQ is not a proper queue, but simply a network library and protocol. How is it even possible to bungle the name so badly and then do nothing about it for twelve years?
gamegod 2 hrs
IDEs - A proper work-grade IDE like Visual Studio will have some tools built into it that will save you tons of time. Features like "Edit & Continue" have saved my bacon many times by making intractably difficult bugs in algorithms much easier to understand because you can experiment with your code on the fly.
There's also some more common features in most IDEs like being able to jump between symbols that saves a lot of time day-to-day.
If I'm interviewing you and you say you don't like using an IDE or a debugger, that speaks to your work experience, your productivity, your self awareness about your productivity, and really puts an upper limit on the difficulty of the problems you've had to solve.
skummetmaelk 1 hr
I find that it is usually the opposite. Devs who don't use big IDEs like visual studio are much more likely to know why things fail in a build pipeline etc.
If you can only build a project by bumbling through menus and pressing a big green button at the end, that is worrying.
If you can only debug by immediately jumping into the debugger and single stepping that is also worrying.
Devs who reach for the tools appropriate to a given situation inspire much more confidence in their abilities.
Knowing the time saving features of your editor is a huge boost. However, the old editors have much more of these ;)
snazz 57 mins
IDEs aren’t useless, you just don’t want them to be a crutch. For this reason, I usually recommend new developers use Nano or a notepad-esque editor until they understand why they might want vi keybindings, then use Vim until they understand why they might want an IDE or something like Emacs. Starting with the IDE hides layers and layers of both junk and useful tools, while experienced developers know which layer of the stack to work on at which time.
clarry 42 mins
If I'm interviewing you and you say you don't like using an IDE or a debugger, that speaks to your work experience, your productivity, your self awareness about your productivity, and really puts an upper limit on the difficulty of the problems you've had to solve.
Or you have surpassed the limit on the difficulty of the problems your debugger & IDE can help with.
asdffdsa 3 hrs
Books: Operating Systems/Database/Networking/Computer Security/Computer Architecture textbooks, Software Engineering textbooks (Clean Code, Design Patterns, Designing Data Intensive Applications, Domain Driven Design as a short list off the top of my head)
lbrindze 1 hr
"Designing Data Intensive Applications" is an absolute goldmine for things like message queues but also going beyond understanding the full implications in database selection and other common, distributed-oriented engineering decisions modern software engineers may come across.
stevekemp 2 hrs
By the same token "documentation". Today I explained to a colleague how they could enable port-forwarding across their already-open-SSH-session via "~C".
Every now and again I pick a tool I use a lot, and read the man-page. Things like "less", "bash", "ssh". Complex enough to contain surprises, but simple enough that you take them for granted.
Almost always this has been time well-spent.
jpmelos 2 hrs
If understanding how Linux works interests you: http://www.linuxfromscratch.org/
danesparza 4 hrs
Caching, logging, centralized configuration, Security, and Design Patterns are all probably easy to overlook.
jedberg 2 hrs
If you're into python, look up decorators and coroutines. They will blow your mind.
Jach 2 hrs
This is still my favorite mind-blowing introduction to them, from 2009: http://www.dabeaz.com/coroutines/Coroutines.pdf Don't have hardware interrupts or threads but still want a multi-tasking operating system? Not A Problem.
nmca 4 hrs
SQL
mfatica 4 hrs
You think a self-taught programmer might never encounter SQL?
politician 4 hrs
Speaking as a technical development manager, I can say that many people have far less than adequate exposure to SQL, and that training people to use SQL effectively and safely is all too often a common requirement before letting them loose on the database.
There are so many ways people misunderstand and misuse SQL and relational databases, it's honestly staggering.
So, I second the OP. Learn SQL, and you'll stand out.
beat 3 hrs
Yeah, learning real sql is a very useful skill. At one employer, I became the "sql expert" because I knew the difference between inner and outer joins. Which, if you know sql, means you know next to nothing. But knowing next to nothing was better than knowing nothing at all, so...
new_guy 4 hrs
'The Imposters Handbook' is good for foundational knowledge, it was written for exactly your use case.
https://bigmachine.io/products/the-imposters-handbook/
LeonB 33 mins
Came here to recommend this too. It’s goal is very much aligned with the OP.
mettamage 4 hrs
It depends on what you want to achieve. I can tell you to read about security but that is only relevant if you want to secure your software. I can tell you about pub/sub, but that is only relevant when you need it.
The things you’ve learned were mostly covered in my computer science curriculum. So I am going to second Robin_messsage his message.
Skim through a CS curriculum.
avip 3 hrs
The concept of monitoring. This simple, obvious concept changed a lot how I think of software maintenance. I usually use slack but that's an implementation detail.
justinholmes 3 hrs
Using GitHub trending for each language look at new things each month.
Build something and think how would I scale this project to X capacity then rinse and repeat.
westonplatter0 3 hrs
SQL
crimsonalucard 2 hrs
Nobody mentions what self taught programmers miss the most.
Theory. And not just algorithm theory.
Zealotux 1 hr
Could you expand on that if you don't mind? I'm a self-taught programmer currently in the process of realising just how much I miss from theory, my list so far on theory fundamentals includes:
• Mathematics and probabilities, and their applications to CS (e.g. formal methods)
• Design patterns (OOP, functional programming)
• Data structures
• Algorithms, time complexity
• System architectures
• Software strategies: CI, CD
• Database principles: SQL
It may sound naive, but I'm kind of overwhelmed by all this and it's not helping my impostor syndrome, I may make a repo of the list with links to ressources I've identified for learning as it seems like a common struggle, maybe it'll help someone.
crimsonalucard 35 mins
None of what you mentioned is theory expect for Mathematics, probability, formal methods and algorithms.
Stuff like design patterns, system architectures and software strategies are like flavor of the week stuff. Opinions basically. Patterns like microservices are bad or good depending on opinion, but theory is always correct. Theory gets less bang for the buck but it's always what many programmers especially self taught ones are missing.
Theory is so hard that it will be hard to see applicability until you're a more seasoned programmer. Many seasoned programmers get by without ever knowing theory. But you will be a better programmer if you know it.
If I were to recommend one theory to study it would be category theory. If there was any true axiomatic theory for design patterns or how to design programs... categories are it. The study of morphisms is the study of the simplest form of a compose-able module. Knowing this theory you will begin to understand why Some design patterns don't work and why it's sometimes hard to reuse patterns in code that was that not properly designed. Theory doesn't answer all questions but for the questions it does answer you will get a definitive answer and not an opinionated one.
malvosenior 3 hrs
If you're coming from Python you should start looking into how other languages handle concurrency. Python has a GIL (global interpreter lock) that only allows for single threaded execution under normal circumstances. Learn about threads, locking, mutexes, semaphores, green threads, race conditions...
mosalarynolife 5 hrs
Design Patterns
groby_b 4 hrs
Just do yourself a favor and read it the way it's intended: As a dictionary.
Design patterns give you a vocabulary to describe design choices you made. They don't give you a set of things that inform design.
wwweston 3 hrs
I can't remember who, but someone's said that design patterns really would be more appropriately called something like "palliatives for static manifestly typed languages."
Knowing idioms that relate to certain expressive / organizational problems is a good thing (especially if you're primarily working in a static manifestly typed language), but there's a weird overcelebrated status to them, and I'm not sure I'd encourage a developer I was training to become familiar with a full catalogue of them.
AnimalMuppet 2 hrs
It seems to me that you could regard something like the Y Combinator as a design pattern that is a palliative for dynamic implicitly typed languages.
All kinds of languages have patterns. Whatever kind of language you use, learn the patterns that are relevant/useful for it.
gwbas1c 3 hrs
A design pattern is a well-known way to do something. It's important so you don't reinvent the wheel.
They're different than external libraries, because usually the details of your application are closely integrated with how you implement the design pattern.
sesser 5 hrs
What are some resources outside of gang of four?
pernambucano 4 hrs
https://sourcemaking.com/design_patterns
jimpudar 1 hr
Patterns of Enterprise Application Architecture
Enterprise Integration Patterns
Just start reading some Martin Fowler books, you will be up to your ears in patterns in no time :)
qazpot 4 hrs
Head First Design patterns
purplezooey 4 hrs
Brown nosing.
