RDS instances in AWS do not get a static IP address. This is usually a good thing, not a problem. This provides flexibility to preserve availability while the physical RDS host may shift around for resizing, or failing over to a different availability zone (AZ). In either case, clients connect to RDS by hostname, and AWS magically updates the hostname to point at the IP address for the currently active host.
Sometimes in Spark you will see code like
In music, you can often write the same note two different ways, for example, B-flat and A-sharp correspond to the same key on a piano keyboard. When you use which depends on surrounding context. A chord C/E/G/B-flat is a C dominant 7th and resolves to an F chord. The same chord written as C/E/G/A-sharp is an augmented 6th and resolves to B major. So which way the chord is written tells you something about where it’s going next.
I have a Lenovo desktop PC at home and some of the diagnostics on the hard drive were failing, like the Targeted Read Test. The drive is a 2TB Seagate.
I found it helpful to create Spark UDFs to make it easier to migrate logic in SQL from another database like SQL Server.
I began to learn Scala specifically to work with Spark. The sheer number of language features in Scala can be overwhelming, so, I find it useful to learn Scala features one by one, in context of specific use cases. In a sense I’m treating Scala like a DSL for writing Spark jobs.
I managed to get Spark to run on Windows in local mode, and to submit jobs to an EMR cluster in AWS.
Working in an agile development environment, I noticed some parallels to my experiences with student theater several decades ago.
Suppose you want to write a report in MSTR to calculate some measure, filtering on patients who had a certain diagnosis and are not on a certain medication. Both Diagnosis and Medication have many-to-many relationships with Patient, and patient count is a metric defined at or below the patient level.
Suppose you have a dimensional model about travel prices like this
AWS is transparent that Redshift’s distributed architecture entails a fixed cost every time a new query is issued. The documentation says the impact “might be especially noticeable when you run one-off (ad hoc) queries.”
I am not passionate about functional programming for its own sake. I am passionate about readability and clean code, and functional programming is a tool to help get there. I take a pragmatic approach: functional programming should be a tool in your toolbox, and you should be ready to use when it makes your life easier. At the same time, I don’t feel a rush to Pure Function and Immutable All The Things. I like languages that support functional programming but don’t strictly require it.
Different languages have different opinions about what to treat as “truthy” or “falsy” when using a non-boolean object as an expression inside an
In PostgreSQL, you can convert child records to look like a nested collection of objects on the parent record. This is useful if you want to convert a relational-style parent-child model into a document style, with the child records represented as a composite within the parent document.
Redux is a library and a pattern for managing state in front-end applications. It is typically associated with React but it can be used with other frameworks as well. Instead of directly modifying state, components dispatch actions, which are then handled by reducers. Reducers take current (immutable) state plus the action to produce a new state. The new state is then wired into React component properties, which triggers re-rendering. This interaction is shown in the diagram below: